Kudos to the OMV gang and ZFS.

  • Just want to thank all the people who make OMV great. After spending dozens of hours messing around with building a 60TB Debian based server and trying to get things to work, hour after hour of searching up commands and firmware and fixing, and tweaking and frustration - I was finally "done". Now I could finally upgrade the hardware on my backup machine, which luckily runs OMV and my first use of ZFS, which hadn't let me down in over a year so far.
    After a couple of hours of setting up the server hardware and installing updates, I threw in the OMV install USB. Going onto M.2 drives, the install took maybe 20 minutes. Booted up, logged in, no errors, no hardware missing, everything was recognized and seemed to be working. That's the Linux I remember. Ran some updates, stopped by the forums to remind myself how to install ZFS for the transfer of the drives, and decided to commit to just pulling the hard drives out of the old machine and throwing them into the new one.
    By now, it was 2am. I just wanted to be finished.
    With the new machine running, I started plopping drives into it. All the while, realizing - Oh no - I didn't keep very good track of which hard drive came from what slot, and with 2 volumes, I'm not sure how I'll match them back up again. MDADM? No idea. I'll have to find a guide or something.
    I figured, well, I'm probably in for a mess to clean up no matter what, so just throw them in and deal with the consequences. I'm sure I'll get it, just don't lose the data. Maybe I can find some way to match them up via UUID or label, hopefully it won't be too bad to deal with.
    Coming from typically using RAID, I had no idea.
    Finished inserting all 12 drives and crossed my fingers that I wasn't in for another sleepless night.
    Logged into OMV, went to ZFS, and decided to give 'Tools --> ZFS import --> Import All --> Import' a try. I cringed and looked away as I clicked.
    DOH. An error. No surprise. Clicked on it to hold it in place while I read it.
    I swear OMV was just playing with me, as the message read in a nutshell, almost sarcastically - "Hey, these pools came from another machine ya know. You COULD use -f to force the import if you want, but you know, it's up to you".
    I knew it. Here comes the final error telling me I'll need to learn the entire structure of ZFS to make this work.
    Back to the menu, check off the 'Import all' and 'Force' buttons and try again.
    I click 'Import'.
    No error message. That's good.
    I then watched as my 2 ZFS pools populated the screen, labelled as I named them, with the correct volume size and usage.
    This can't be.
    Is it really this easy? Did it seriously work on the first try? Did it seriously not care where in the array the drives were located, and it just matched them up anyway?
    Set up my shares and checked everything. It was all there.
    The process took all of about 30 seconds.

    Damn. Thank you OMV. After all the headaches of the past week of setting up my main server, this was the one redemption for Linux (Debian especially). And thank you to the people that made OMV what it is. If only I could have a desktop environment to run Kodi directly off my server to my theatre, I'd be using OMV for every computer from now on. But I'll take what I can get, and thank you for that.

    • Official Post

    Look at the drives in the pool and then get the serial number of the disk from the Storage -> Devices tab. Next time the server is off, slide the disk out and find the serial number on the label.

    f only I could have a desktop environment to run Kodi directly off my server to my theatre

    spend the money on a firetv stick and sideload kodi on it. It will be better at video and use a lot less power.

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • How would it be better at video over a network than a dedicated GPU direct from drives without transcoding?

    I should probably qualify that - I don't like using stripped down compressed video files. I prefer raw ISOs direct from discs. This is an effort to move away from using Kodi on an Xbox Series S over wired 1Gbe.

  • I have Plex docker running server side then on my TV I just install Kodi on my Xbox Series X and stream direct 4k flawless.

    See, that's the thing. As soon as it's transcoded, I'm not so sure it's flawless. Just compare Netflix or Amazon "4k" to a good local copy of the same video and you'll see what difference it makes.

  • Look at the drives in the pool and then get the serial number of the disk from the Storage -> Devices tab. Next time the server is off, slide the disk out and find the serial number on the label.

    Just noticed this part of your response - I've done that in the past with OMV - and noticed there's some inconsistency in where you'll find the actual serial number that's on the label, depending on whether you're looking at the Devices tab or in the S.M.A.R.T. information - they're often different from each other. Maybe it's due to using previously owned drives or something, where they've been dd'ed or something?

    For example, my current drives:

    From Devices under Disks:


    /dev/sdc
    ST8000NM0055-1RM112
    ZA11TJ7L
    ATA
    7.28 TiB

    Same drive under SMART:


    /dev/sdc
    ST8000NM0055-1RM112
    ATA
    ZA128D0Q
    7.28 TiB
    32 °C

    • Official Post

    How would it be better at video over a network than a dedicated GPU direct from drives without transcoding?

    Amazon spends a lot of money creating a highly optimized android and cpu/gpu combo that uses almost no power. I have a firetv on every tv in my house. It can play every video with no transcoding I have thrown at it without issue. Having some big gpu in your server has more power but it is just wasted. Most of my firetvs are using the wired adapter. It is only 100mbps but that is fast enough for 8k 60hz video. I have had 6 TVs/firetvs playing 4k video over nfs at the same time without a glitch and my server barely even noticed. An RPi4+ running kodi would work very well too.

    I don't like using stripped down compressed video files. I prefer raw ISOs direct from discs

    I have played dvd and blueray isos on kodi with firetv without an issue.

    See, that's the thing. As soon as it's transcoded, I'm not so sure it's flawless. Just compare Netflix or Amazon "4k" to a good local copy of the same video and you'll see what difference it makes.

    I don't have an xbox but kodi on firetv and rpi or just about any arm sbc or intel cpu with integrated gpu don't need transcoding.

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post

    I've done that in the past with OMV - and noticed there's some inconsistency in where you'll find the actual serial number that's on the label, depending on whether you're looking at the Devices tab or in the S.M.A.R.T. information - they're often different from each other. Maybe it's due to using previously owned drives or something, where they've been dd'ed or something?

    dd can't change the serial number. It is very difficult to change the serial number of a hard drive. No one can accidentally do it.


    I haven't seen the serial number wrong on mine. I have four different manufacturers in my server and all 10 are correct in the Disk tab, smart, and drive label.

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • dd can't change the serial number. It is very difficult to change the serial number of a hard drive. No one can accidentally do it.


    I haven't seen the serial number wrong on mine. I have four different manufacturers in my server and all 10 are correct in the Disk tab, smart, and drive label.

    I'll have to check to see which one's correct. Judging from the SMART details and the hours presented, I assume the SMART information is more likely to be messed with, but more likely to be obvious?
    I can only guess at why they'd be showing 2 different serials.

    • Official Post

    I assume the SMART information is more likely to be messed with, but more likely to be obvious?
    I can only guess at why they'd be showing 2 different serials.

    the OMV disks tab gets the serial using udev. But smart and udev both get the serial from the disk's firmware. I don't know how either would be wrong.

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post

    Well that's crazy then. I've got several disks like that.

    Any of them have a sata/sas expander or usb or hba in front of them?

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I do not do ANY Transcoding.... I play DIRECT PLAY from Server to Kodi, Office to TV via CAT8 LAN.

    Transcoding, I noticed, when doing a 4k Movie to TV, actually downgrades. All my movies are 40-90GB 4K Movies, I do not Transcode unless I am viewing remotely.

  • I do not do ANY Transcoding.... I play DIRECT PLAY from Server to Kodi, Office to TV via CAT8 LAN.

    Transcoding, I noticed, when doing a 4k Movie to TV, actually downgrades. All my movies are 40-90GB 4K Movies, I do not Transcode unless I am viewing remotely.

    Maybe it's my knowledge being dated, my understanding was that as soon as you stream over a network, transcoding takes place. I never really looked into Direct Play on Plex, as the only time I ever use it is when I'm on a TV without Kodi.
    Transcoding absolutely does exactly that, it compresses the data. I even avoid .mkv conversion though, I rip straight to .iso whenever possible. Not to say I don't have .mkv's, but when I compare both the .iso and a 'high quality' .mkv, there's still a difference.

    • Official Post

    They all would, they're in 12 bay servers.

    Depending on the device used, it can not pass a serial number or pass its own "serial number" or possible mix them up.


    Maybe it's my knowledge being dated, my understanding was that as soon as you stream over a network, transcoding takes place

    Maybe with plex but not kodi streaming directly from nfs.


    Transcoding absolutely does exactly that, it compresses the data. I even avoid .mkv conversion though, I rip straight to .iso whenever possible. Not to say I don't have .mkv's, but when I compare both the .iso and a 'high quality' .mkv, there's still a difference.

    DVDs and Bluray are compressed already. dvds are mpeg2 and bluray is usually mpeg2 or h264. mkvs are usually h264 or h265. I get it that you have more or less aggressive compression from those protocols. I re-encode my rips to h265 mkv. I doubt most people would be able to tell a difference in a test where they didn't know which is which.

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Depending on the device used, it can not pass a serial number or pass its own "serial number" or possible mix them up.

    Device in terms of hard drive, or device in terms of SAS/SATA/RAID controller (keeping in mind, I really haven't figured out how those controllers interact yet, my one server has 2 different controllers, which really upset my limited understanding)?


    Quote

    Maybe with plex but not kodi streaming directly from nfs."

    I'm really not sure about Plex, didn't know that about Kodi. My brain is still working with what information I dug into 15 years ago lol. Haven't really studied up on it since.
    As for NFS, sounds like I need to learn more. I've messed with NFS, but with it being more troublesome than SMB I just stuck with SMB. I had read somewhere that it wasn't as fast as SMB, and with SMB multichannel I've found more reasons to stick with it.

    Quote

    DVDs and Bluray are compressed already. dvds are mpeg2 and bluray is usually mpeg2 or h264. mkvs are usually h264 or h265. I get it that you have more or less aggressive compression from those protocols. I re-encode my rips to h265 mkv. I doubt most people would be able to tell a difference in a test where they didn't know which is which."

    Agreed, but I don't see the benefit of compressing it more. Unless I learn of a lossless encoder, I'll stick with the ISO's as much as I can - looking forward to the day that BD menus can finally function in Kodi too lol. Obviously I avoid Plex for the fact it doesn't handle ISO's at all.
    My understanding from TrashGuides is that h265 is an inferior format to h264. As for telling the difference, no, most people don't even look at picture quality, period. I find myself distracted from the movie when a dark or bright scene, or a scene with a lot of sky in the background come up and you see pixelation.

    • Official Post

    Device in terms of hard drive, or device in terms of SAS/SATA/RAID controller (keeping in mind, I really haven't figured out how those controllers interact yet, my one server has 2 different controllers, which really upset my limited understanding)?

    the sas controller.


    I'm really not sure about Plex, didn't know that about Kodi. My brain is still working with what information I dug into 15 years ago lol. Haven't really studied up on it since.

    nfs or smb served files have never been transcoded by kodi. It really doesn't make sense for the client to transcode since the bits have already been transferred.

    As for NFS, sounds like I need to learn more. I've messed with NFS, but with it being more troublesome than SMB I just stuck with SMB. I had read somewhere that it wasn't as fast as SMB, and with SMB multichannel I've found more reasons to stick with it.

    For me nfs is easier. nfs is less overhead as a protocol than smb. I can saturate 10gbe with nfs. I don't use windows and smb multichannel only helps multiple file transfers.


    Agreed, but I don't see the benefit of compressing it more

    Less storage, less hardware cost, faster backups, faster moves to new storage, less network bandwidth all seem to be benefits.

    My understanding from TrashGuides is that h265 is an inferior format to h264. As for telling the difference, no, most people don't even look at picture quality, period. I find myself distracted from the movie when a dark or bright scene, or a scene with a lot of sky in the background come up and you see pixelation.

    I want good picture quality and what I have been doing works for me. Other than patent stuff and more cpu required (if gpu doesn't natively decode it) for h265, I don't know why it is inferior. If you want menus, keep using your method. I have been ripping my collection to my server for almost 20 years. Not going back to redo anything for a few pixels.

    omv 8.1.1-1 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.7 | compose 8.1.5 | cterm 8.0 | borgbackup 8.1.7 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!