[HOWTO] Instal ZFS-Plugin & use ZFS on OMV

  • I have the N54L with 16GB. I tested it with 4 x 4TB drives in RAIDZ2 and the memory never maxed out.


    For a 4 disk array in RAIDZ2 of 2TB drives, 8GB is more than enough to run ZFS and do scrubs. There are plenty of people with 16GB with way more drives. The manual may say use a gig per TB, but for your N40L and your 2TB drives you'll be fine.


    The only area you may struggle with is deduping, which if you do not use won't be a problem for your CPU.

  • @thespeed I know that you're a drive down because of the issues you've had with Xpenology, but just some friendly input to stop you suffering further data loss -


    Thanks for the answers.


    I have bought the system second hand a year ago and i have checked the S.M.A.R.T data.
    15842, 15200 and 17985 hours.


    So with some calculating the system runned for +- 2 years, 24 hours a day.


    It's a shame that it takes a lot of space to run Raid Z2 or Raid 6.
    Maybe it's better to add a bigger new drive and go with Raid-6 or Raid-Z2


    @ellnic
    Are you running Raid-Z with Openmediavault with no issues,
    so is it a safe option?


    I am always forgetting to make backups. :S
    But i am going to automate that with an external USB drive.
    Better safe then sorry.

  • Safe is a tricky thing to guarantee when it comes to data.


    What I can tell you is:


    1. ZFS on Linux is the underlying technology that the plugin interacts with, so you can use ZFS even without the plugin. Let's assume that for a moment the plugin stops working or OMV no longer supports ZFS (not gonna happen, but for the sake of argument) you can still issue commands in a terminal to manage ZFS. ZFS has been available for Linux for a while now, but the OMV plugin makes it as easy as the BSD solutions (FreeNAS/NAS4Free) with the massive benefit of the Linux system (hardware support/Debian packages etc) and not having to install to a key drive.


    2. The plugin has certainly been stable for my use on the N54L and I am in the process of building a replacement server which will also use OMV and the ZFS plugin. I will be using 8 x 4TB drives in a RAIDZ2.


    3. If things go wrong - it is no more or less likely than them going wrong on BSD, and it probably related to the actual drives rather than the system that has been managing them.


    With regards to adding more drives to your system:


    Keep the drives the same size. If you have 2TB, stick to that. ZFS pretty much requires this. There are some nasty split partition 'work arounds' but I wouldn't call them solutions.


    RAIDZ1 = one drive for parity (let's say lost space for this example)
    RAIDZ2 = two drives lost space
    RAIDZ3 = three lost space


    This is the same whether you have 4 drives or 18.


    So to run RAIDZ2 with 4 drives, you lose 2. To run it with 6, you still only lose 2. To run it with 18, you still only lose 2. 3 for RAIDZ3, 1 for RAIDZ1.


    So, to increase your storage for smaller budget setups you are better going for lots of smaller drives than bigger ones.


    For example: 4 x 8TB drives = 32TB, but in a RAIDZ2 you lose 2. So 16TB usable space. Whereas, if you were to use 8 x 4TB drives which is still 32TB as they come, you only lose 2 of them with RAIDZ2 leaving a much greater 24TB.


    If you intend to stick to 2TB drives in RAIDZ2, you could get a decent eSATA card (the internal one sucks for speed) and put 2 more 2TB in a 2 bay eSATA enclosure. This would give you 8TB.


    You need to find the right balance of price to space and redundancy ratio for your budget. :)


    And don't forget your backups. ;)


    Btw, have you heard of HDAT2? You may want to try running that on all your spindle drives (don't use on solid states).


    Edit: sorry if I am bombarding you with info here, but also: bare in mind that once a ZFS array is created, you cannot add drives to it (or take them away) unless you destroy and recreate it, which means backing up all the data. This is one of the current problems I am facing. I am building a new box that will last me some time. It's pretty much way exceeded the original budget which is no fun :-/ but it will last a while.

  • Really nice to hear that the plugin is working so well for you. Regarding support, I still don't have that much free time due to the current family situation but it might change sometime in the future :) Until then I would definitely be available to help out anyone who want to continue working on the plugin if there are any questions on the current implementation. The code is available on GitHub for anyone interested :)
    One thing that I think should be investigated is how to bump the dependencies on the ZFS packages (how smooth it is to update to the new 0.6.4.1 release). I haven't done this myself yet, but will probably try it out quite soon...

    • Offizieller Beitrag

    One thing that I think should be investigated is how to bump the dependencies on the ZFS packages (how smooth it is to update to the new 0.6.4.1 release). I haven't done this myself yet, but will probably try it out quite soon...


    It should update automatically unless the repo location changed. The dependencies in the control file don't specify a version.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!


  • It should update automatically unless the repo location changed. The dependencies in the control file don't specify a version.


    That's true. I thought we had specified the 0.6.3 version in the control file, but that was wrong :) Guess I just have to upgrade my system then to test how the ZFS package update behaves...

  • @ellnic


    Thanks for all the information
    I have just bought a new WD Red 2TB drive for my server and installed it.
    Also i looked at my existing seagate barracuda st2000 drives and found out that it isn't the most reliable drive.


    I updated the firmware of the drives and made a raid-Z2 pool.
    Everything seems to be working okey :)
    So now i need to install all the plugins and the other stuff.


    At what kind of interval do i need to do a scrub?
    Don't know exactly how to make a good cronjob for that.


    I think i will stick to the standard SATA ports at this moment and have installed the patched bios.
    So all the Sata ports runs at 3GB/s
    If it's to slow then i can always buy a different controller :)

  • @thespeed glad you got everything up and running :)


    Am I right in thinking that you now have 4 x 2TB drives in your RAIDZ2 array, so 4TB usual space? For this you could scrub once a month would be suffice, and it'll probably take a few hours. If you have your NAS on all the time, you can use the 'Scheduled Jobs' in OMV to create a cron job for it, I personally just scrub manually when I know the server will be on and in minimal use. This obviously relies on me not forgetting, but then I monitor things pretty closely.


    The cron job can be setup using the Scheduled jobs, specifying all the time info and the command of: zpool scrub [name]


    So for example:


    Enable: tick
    Time of execution: choose
    User: root
    Command: zpool scrub media


    Choose if you want the results emailed and comment if you want in the box.


    Glad you got the modded BIOS flashed - full speed ports and hot swap are just a few of the benefits. It also enables port multiplier on the eSATA port should you decide to go down that route. There were some BIOS settings that needed changing, but it's possible. :)

  • I'm using the latest version (omv 1.9 with all updates) of the zfs plugin and have imported a pool with many datasets and they all have many many snapshots (e.g "zfs list" return a few screen full of text).


    When I try to add "Share Folders" it timeout whilst trying to make a list of volumes for me to select. The error msg that comes up is "An error has occurred" then under details->"communication failure". I'm guessing that the default "timeout" is not long enough for it to make the list.


    A previous version of omv (not sure which of the sub 1.9) seems to list the volumes in reasonable times. But now it's no longer works. I remember reading something about the zfs plugin developer suggesting an improvement to that makes it faster / not hang.


    The other old threads with timeout issue seems to not apply to the latest version of omv webgui. Where how do I increase the timeout so to give it enough time to make the list so I can create shares. ?(


    Thank You.

  • I don't have a solution for this unfortunately. I would guess that the timeout is specified somewhere in the "core" code of OMV. I'll have to test setting up a similar scenario as you describe in a virtualized environment to see if I can reproduce the problem. However I won't be able to do this next couple of days.


  • The lastest version of OMV isn't 1.9. It's 2.x. In fact, as of yesterday, 2.0.7. http://www.openmediavault.org/?p=1682


    I'd update and see if this fixes your issue.

  • And I wouldn't update to 2.0 because 1.0 still receives bugfixes, if neccessary.


    2.0 is testing.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • I've never seen someone mention something like that, ever, before.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • No longer a good idea? The article mentions a problem every, what, 12TB?
    Maybe I lack a full understanding of the problem but to me it doesn't sound like that kind of problem seems to affect even 1TB hard disks?!


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Forgetting the figures for a moment - the way I read it was purely that RAID5 and RAIDZ1 arrays can put you in a tight spot when a drive fails - you've pretty much got a potential (likely?) hairy situation because drives don't seem to last now days.


    Unless you've got a backup... Which really should have been the point pushed.

  • No longer a good idea? The article mentions a problem every, what, 12TB?

    He is talking of hardware specs, not rumors.
    URE or Unrecoverable read errors is specs from hard drive manufacturer. https://en.wikipedia.org/wiki/RAID#URE
    It is the rate of natural data corruption in a hard drive. Every 12 TB read you get something that is corrupted. The URE is more or less the same number even in the past. But since hard drive capacity increases, the chances of seeing them is higher.


    This is the main reason behind ZFS, btrfs, that crappy server-grade fs from MS, RAID5-6 and Snapraid (weird trickery but does the job nontheless, especially on weak ARM hardware like my Zyxel NSA325v2).
    Because if you get that you lose data. Checksumming and parity avoids that.
    http://blog.fosketts.net/2014/…-data-integrity-checking/


    With a working RAID5 (or ZFS equivalent), that is scrubbed regularly the controller pulls the data from the redundant stripes, and fixes the issue. Assuming a decent controller.


    With a RAID5 where a drive has failed, it is a BIG issue because if you get an URE while rebuilding it the controller has no parity to fix the error.
    Good controllers stop and tell you of the issue so you can move the data off it, bad controllers ignore it and happily corrupt the data.


    Since we are talking of probabilistic events, you don't know when a specific drive will reach its mark and throw a URE.
    But if your RAID5 array is 12+TB that's very likely you will get one, because that's 12TB of data read.
    If the array was 6 TB, you have 50% chance of getting a URE when rebuilding a RAID5.


    Of course if you are into big storage businness like say making a 30+TB storage, in case a single drive fails you will surely get an URE on rebuild, and someone will start shouting at you.


    There are two different solutions to this technical issue.
    - buy enterprise-class SAS or Sata drives that have an URE an order of magnitude higher, so you will get an URE once every 120TB of data read. But that's expensive.


    - use RAID6 or ZFS 2-parity equivalent. Since an URE is a small-size data corruption, it's highly unlikely to happen twice on the same data (both on data X and in the parity of data X), so you are safe no matter how much URE you get during rebuild.


    Eventually, you will need moar parity.


    Zitat von ellnic

    hairy situation because drives don't seem to last now days.

    That's obvious and expected. Hard drives have much larger capacity than back then, in the same footprint. What does this mean? that the traces on the platters are more and more small, and the rest of the hardare is more and more sensible. More complexity = more fragile.


    For example, the cutting edge of HDD tech is filling the drive with helium (less drag on platters and drive head). 8 and 10 TB per drive.
    In the near future HAMR technology (heat assisted magnetic recording) will bring you even higher TB per drive. They will work by using a tiny laser to heat up the platter surface in front of the reading thing so the trace gets bigger (due to metal expanding with temp rise) so the reading thing can read it, this ridicolous overengineering is not going to make HDDs more reliable.


    Zitat von ellnic

    Unless you've got a backup... Which really should have been the point pushed.

    RAID was invented for better data availability and higher server uptime, not for backup. So the article focuses on strategies to keep enjoying better data availability and higher server uptime before mentioning that yeah, for home use is ok if you have backups.


    If when a hard drive fails in a RAID5 and you get a URE on rebuild, the whole array has to be erased and data restored from backups (taking ages because tapes, or clogging bandwith from another server to restore the data), that's a server that goes down for hours at a random moment, and someone will be pissed.
    No data loss of course, but the entire point of RAID 5 or ZFS single parity was to avoid downtime.
    If the reaction to disk loss is "restore backup" I'm better off with something like AUFS or by just splitting manually the data over multiple drives and checksumming them with a bunch of scripts.


    Besides, that's a semi-pro article for semi-pros, if you look at the original article it references, you see it was written in 2009. Not a lot of people was building a NAS with 12+TB of storage back in 2009.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!