Is ZFS supported in Kernel 4.13-4.15?

  • It’s usually seconds. I’ve upgraded all of mine.


    I think the concern is that if you upgrade: will you always be able to get those drives in a system of at least that version? Probably. There were no additional upgrade prompts on 0.7.9 for me, so it’s the previous version that prompted. I think we’re at the point now where we can get to 4.16 and 0.7.9 easily.


    Do it :-)

  • I did the "zpool upgrade" some time ago: the process is a matter of minutes (even less) if I remember correctly and everything went
    well without issues. See post #69.


    OK, thanks for the hint to post #69. I didn't remember that you already answered my question. ;)


    It’s usually seconds. I’ve upgraded all of mine.
    I think the concern is that if you upgrade: will you always be able to get those drives in a system of at least that version? Probably. There were no additional upgrade prompts on 0.7.9 for me, so it’s the previous version that prompted. I think we’re at the point now where we can get to 4.16 and 0.7.9 easily.


    Do it :-)


    Today in the morning I recognized a known issue with my zfs file systems within the section "Access Rights Management - Shared Folders". My solution I described here:


    ZFS device(s) not listed in devices dropdown



    But OK, after solving that, I did the "zpool upgrade" and it worked fine within under 5 seconds. ;)



    Thanks for your help. It's much appreciated.


    Regards Hoppel

    ---------------------------------------------------------------------------------------------------------------
    frontend software - tvos | android tv | libreelec | win10 | kodi krypton
    frontend hardware - appletv 4k | nvidia shield tv | odroid c2 | yamaha rx-a1020 | quadral chromium style 5.1 | samsung le40-a789r2
    -------------------------------------------
    backend software - debian | openmediavault | latest backport kernel | zfs raid-z2 | docker | emby | unifi | vdr | tvheadend | fhem
    backend hardware - supermicro x11ssh-ctf | xeon E3-1240L-v5 | 64gb ecc | 8x10tb wd red | digital devices max s8
    ---------------------------------------------------------------------------------------------------------------------------------------

    Edited once, last by hoppel118 ().

  • What do you recommend?


    I recently upgraded my ZFS OMV3 to 4. This was before i saw that ZFS didn't work with the latest kernel. The reason I upgraded to begin with was my system somehow became read only. I couldn't figure out how to fix.


    I didn't export my ZFS pool. But i still have my original OMV3 USB install stick.


    Do you recommend i hook that up first to export? Or keep my OMV4 SSD hooked up and import?


    I also don't remember what i named my pool. So can i even import without a export and a name?


    This is the first time doing this and am kind of stuck/nervous on what's next step to take.


    Thank you!

  • It does work with the latest kernel. It just doesn’t work with the latest kernel/version of ZFS combo in Debian’s own repos - this will change - soon.


    In the meantime you can enable OMV extras testing to get the latest ZFS version, which works just fine with kernel 4.16. It’ll take you 10 minutes tops.


    You should ideally export a pool before moving it to a new OS, but it’s not essential if you absolutely know it’s not in use elsewhere. It’s just not best practice to skip that step. You can use the -f flag to force import. You don’t need to remember the name of the pool if you’re importing. You can easily find out... Just use:


    Code
    zpool import


    to see what is available to import.


    Once you have the name of it, you can:



    Code
    zpool import -f poolname


    Or there is an option in the GUI to force. You can do all of this via GUI, infact. There is a switch to import all available. You’ll want that if you don’t know the name. You’ll also most likely need to force import as ZFS will think it’s still in use on another system if you didn’t export.


    I would highly recommend finding out why your ‘system’ became read only. I assume you mean your pool was read only? I.e. a permissions problem. You could have fixed this in 20 seconds with chmod and chown. If it is a permissions error, nothing will change in OMV 4... your pool won’t automatically become owned by your new systems user account. You need to issue commands to fix/alter it.


    Get up and running and get your pool imported first then post again if it’s still read only and we’ll try and sort it.

  • I really wish we could have switched to Ubuntu for the base OS...

    A few years ago I would have said the same things. Now, Ubuntu 16.04 LTS is just as stable as Debian.

    If Ubuntu is based on Debian, I'm guessing there are differences are in how repositories are maintained?


    I suppose the question is, what are Ubuntu maintainers doing with the Debian base, that Debian maintainers are not doing? Tweaking, regression testing?, etc.

  • I'm guessing there are differences are in how repositories are maintained?

    Yep and the release schedule is different.


    hat are Ubuntu maintainers doing with the Debian base, that Debian maintainers are not doing? Tweaking, regression testing?, etc.

    I don't consider ubuntu more stable than Debian but Ubuntu has added features that Debian doesn't add/accept. zfs being part of the kernel package is one thing. Supporting kickstarting via debian installer is another. The backports repo is quite a bit different especially where the kernel is involved. The LTS releases truly are supported for five years where Debian falls off to a subset of the packages later on. Ubuntu seems to put newer versions of packages in the LTS releases while maintaining the same stability as Debian as well. And if you want commercial support, Canonical's support is quite good (many times better than Redhat).

    omv 5.5.2 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I don't consider ubuntu more stable than Debian but Ubuntu has added features that Debian doesn't add/accept. zfs being part of the kernel package is one thing. Supporting kickstarting via debian installer is another. The backports repo is quite a bit different especially where the kernel is involved.The LTS releases truly are supported for five years where Debian falls off to a subset of the packages later on. Ubuntu seems to put newer versions of packages in the LTS releases while maintaining the same stability as Debian as well. And if you want commercial support, Canonical's support is quite good (many times better than Redhat)

    Since OMV is designed as a server, those are compelling arguments for adopting Ubuntu for the base. While it's not my area of expertise, I imagine using their LTS repo would result in fewer "odd ball" repo problems, brought on the forum, over the long haul.

  • Hi there ;)


    So, I was wondering if I can go with the ZFS update to v0.7.9 with my 4.14 kernel as I can see the ZFS package are available ?
    I have the linux-headers installed for that kernel.



    Do you see any risk ?
    If that doesn't work, can I rollback if I keep my current conf available at boot ?


    Thx


    P.S : Don't you think we could add a specific zone in the forum for all the ZFS stuff, like for RAID or in the Configuration section ? This could be a good improvment no ?

    Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
    1x500MB SSD for System/Caches/Logs/Backup/Downloads | 7x2To HDD RAID 6 for Datas
    Powered by OMV v4.1.29 / Kernel 4.18

  • @sbocquet to my knowledge, older kernels with newer ZFS versions is not an issue. It was just an issue with 4.16 needing 0.7.9.


    @flmaxey A couple of years ago I wouldn’t have considered Ubuntu as an option for anything critical or stable.. in recent years that has changed. I’ve been thinking quite a bit about it over the last couple of days and the more I think about it, the more I think Ubuntu may actually be a better choice for OMV 5.


    Some boring stuff you probably don’t need to know but led to my original opinion of Debian/Ubuntu, and my opinions now. I’m a multi OS user but prefer Mac OS. With Linux, I’ve tried a ton of them. I first toyed with Linux back in the days of yellow dog. It was a rather miserable experience, and involved a lot of manual everything. Bearing in mind that yellow dog was for PowerPC chips, I was left wondering what the hell I was doing playing with a piece of junk like that when System 8 was so much more pleasing to use, despite its flaws.


    I left Linux alone for quite a while and revisited many years later with Ubuntu, back when they used to drop CDs in the post free of charge. I liked Ubuntu because it was very different from other distros in terms of its ease of use - getting up and running, and friendly user base. I owe Ubuntu for getting me really interested in the benefits Linux could offer and for a time I stuck with it. After a time, I tried Debian and it was like a breath of fresh air in terms of stability.


    Linux frustrates me at times... especially when I look at the fragmentation. In many respects, I agree with Bryan Lunduke and his Linux Sucks spiel. In reality, there are a ton of amazing fragments of Linux, and only a tiny handful of distros actually worth considering. These boil down to:


    • Workstation distros - the most up to date packages in the repos, great GUIs and stable - assuming that you want a bit more than a ‘just functional’ workstation. Personally, I love pleasing GUIs. That’s why I love MacOS.
    • Server distros - rock solid, easy to administer/maintain - who really cares what it looks like, you probably don’t spend all day logged in anyway


    Things to consider:


    • Developer/user base
    • Commercial support or backing
    • Use or avoidance of anything non FOSS
    • A possible crappy package manager
    • Amount of packages out the box unless you want to compile everything


    For a workstation, Debian isn’t my ideal choice. It’s solid, but solidity in the Linux world often comes at a price of old out-of-date packages. Debian is, however great for servers. Sure, you can use testing or sid... then it’s not stable. Fedora is bleeding edge.. but it’s unstable and is mostly FOSS. Why they push that for servers is beyond me. Any distros I try, I really try to like. And I think I’ve experienced ridiculous out of box bugs with Fedora more than any other. That brings me to things like Arch and derivatives. Excellent for workstations if you want the best of Linux but can be problematic if you don’t update it every 5 seconds (and even if you do)... not good for servers. Both Debian (and derivatives) and Arch (and derivatives) have great package managers though. Something which a ton of other distros still lack, quite sadly. Anyone would think we’re still in the 90s.


    Looking at the current top 10 on distro watch over the last 12 months, I see the following and here are some very incomplete, very biased and probably very shallow thoughts on them:


    • Mint - Urgh. Thanks for cinnamon, I think.. wait.. I’ll have KDE. Well, thanks for Mate, actually no I’ll take XFCE. Also, I can’t use a distro called Mint. Why don’t people think about the names they call their projects? No one is going to take you seriously when you say your servers run this... or workstations.. or anything. What distro do you use? Werther’s Originals. Also, small dev base compared to the likes of Debian
    • Manjaro - I love Manjaro, but not for servers. For the most part they seem to be getting it right. Glad to see the project doing so well. Nice extras for noobs too, like Kernel chooser GUI. It would be fair to say they’re also a small dev base, but then most are when compared to Debian.
    • Debian - the mother/father/great uncle/cousin etc of Linux as we know it. Epic package manager, tons of packages, easy to admin and maintain, HUGE dev team and user base. Everyone makes deb binaries available for their projects. Solid solid solid. Auto config/setup on package installation. Not too up to date though, without moving branches, which sacrifices stability. Silly thing is, the volatile nature of sid (that you would need to get the newer packages) actually makes it less stable than Arch based distros. Easy to package for. Easy to host repos.
    • Ubuntu - we all know it... money makes things move. Does this mean there is no good FOSS without corporate backing? Of course not. But money makes development move. Ubuntu is a lot more stable than it used to be. They seem to have got their silly ideas (Unity!) out of their heads and are moving back in the right direction. I was surprised to see Ubuntu as a supported OS on HPs site for the Gen 8 microserver alongside Red Hat and yet not Debian. They’re pushing themselves very heavily as a commercial solution. A ton of Debian’s plus points and now that stability has improved, very little to complain about. Better LTS model, newer packages, commercial support if needed. PPAs, uses apt and debs. Seems to be providing the goods... took a legal risk and won... ZFS.
    • Solus - I’m not too familiar with it but I’ve read the package manager sucks and it’s MORE fragmentation. Budgie looks nice, but it’s more dilution of coding efforts in the community.
    • Antergos - call me shallow but I hate the name. Its gross. ;-) It’s arch though, so pretty much all the plus/negative points that brings with a GUI installer. I think I prefer Manjaro.
    • Elementary - What? NEXT!
    • Fedora - Red Hats loyal group of butt monkey Guinea Pigs. Yeah you get cutting edge. You also get bugs bugs bugs. Limited to FOSS without additional repos which can, and often do, break things. Yum is slow and sucks. Dnf is better, but is still slow and inferior to apt. Dnf and yum do have one of the nicest/tidiest outputs though and I really wish apt would look like this.
    • OpenSUSE - very very very easy to administer but YAST will make you lazy. Things do tend to work very well out the box with SUSE, though. They’re also totally in bed with BTRFS and seem to be anti ZFS. I haven’t used it a lot in recent years but it’s good to have both tumbleweed and leap options. Another thing I noticed was that YAST in openSUSE is not the same as SLE. It seems to be missing a lot of modules. SLE is much faster than openSUSE. So openSUSE is more like Fedora than Centos. It’s still stable though, the best RPM distro IMO. Thanks for the open build service.
    • TrueOS - not Linux.


    There’s only 2 distros on that list that I would want to use for a server. That’s Debian and Ubuntu. SUSE is ok if you don’t need ZFS and don’t mind compiling a lot. More devs offer apt packages and even Red Hat than SUSE. Not a massive issue, I know. But it’s all the little things.


    Other mentions:


    • Centos - Redhat, more out of date than Debian a few years in. SLOOOW release cycle. Red Hat - and now owned by Red Hat.. not really Linux. It’s a business.
    • Oracle - just.. no.. really. Just no. Just release ZFS then disappear. And take Java with you too.
    • Source Based (Gentoo etc) - love using them, great to tinker with but so impractical and time wasting unless you make custom digital juke boxes or kiosks.
    • Unix - limited hardware support
    • All the other distros where they’ve chucked their choice of packages together with a nice theme and a crappy name - why?!


    So, IMHO, Ubuntu and Debian are the only choices for a decent solid server OS. OMV 5 must have ZFS. Ubuntu makes this easy and would eradicate a lot of the issues we see installing / updating Debian and keeping ZFS ticking. It may well be the better choice.

  • I only use Debian as server (work and home) and Windows/Mint LMDE as Workstation (home), and I just can be agree with you... OMV need ZFS as a 'native' storage solution.


    I work all days on Solaris 11 with native ZFS storage, and I cannot go back to RAID/ext 4 without feeling 'in danger'.
    File checksum is a must have, and so the snapshots... They solve some of my problem before it happens !

    Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
    1x500MB SSD for System/Caches/Logs/Backup/Downloads | 7x2To HDD RAID 6 for Datas
    Powered by OMV v4.1.29 / Kernel 4.18

  • The zfs 0.7.9-3 packages are in the stretch backports repo. So, I have removed the 0.7.9-3 packages I built from the omv-extras testing repo since they are pinned higher and I prefer the Debian built packages be used.

    omv 5.5.2 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • What do you mean by native?

    I know Debian cannot have ZFS directly after finished the install (and so OMV), but ZFS is a must have when you come to NAS storage.


    Maybe the Ubuntu way (and don't get me wrong, I really prefer Debian) should be consider as ZFS should be a basic feature out of the box for a NAS.
    I clearly don't speak for myself as I work all day on computers, but let's think of basic users.


    On the other hand, the ZFS plugin is doing some really good work for that... but when problems comes, and they come, you have to know what your doing. Plus you have to update/maintain the plugin...

    Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
    1x500MB SSD for System/Caches/Logs/Backup/Downloads | 7x2To HDD RAID 6 for Datas
    Powered by OMV v4.1.29 / Kernel 4.18

  • On the other hand, the ZFS plugin is doing some really good work for that... but when problems comes, and they come, you have to know what your doing. Plus you have to update/maintain the plugin...

    A lot of the zfs plugin problems are due to the kernel and zfs versions. Moving to Ubuntu would definitely fix that. Even if zfs was a core feature (not going to happen), it would still have to be maintained. If OMV 5.x is designed around btrfs, I think it will be easier to make zfs "happy".

    ZFS is a must have when you come to NAS storage.

    I don't use it :)

    omv 5.5.2 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • A lot of the zfs plugin problems are due to the kernel and zfs versions. Moving to Ubuntu would definitely fix that. Even if zfs was a core feature (not going to happen), it would still have to be maintained. If OMV 5.x is designed around btrfs, I think it will be easier to make zfs "happy".

    I don't want to be rude, but please, save us from BTRFS ;)


    Honestly, I don't mind using one or the other, I'm not a church preacher. I just want something that works and secures my datas; but in 2018, BTRFS development is near death ;)...
    If Oracle (and I don't like them too ;) ) use ZFS in Solaris, believe me, it's because it's rock solid, unlike BTRFS. ZoL is probably not as solid as ZFS on Solaris, but clearly, it is far ahead.


    https://www.ixsystems.com/blog/open-zfs-vs-btrfs/
    http://blog.fosketts.net/2017/07/10/zfs-best-filesystem-now/


    We will see...

    Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
    1x500MB SSD for System/Caches/Logs/Backup/Downloads | 7x2To HDD RAID 6 for Datas
    Powered by OMV v4.1.29 / Kernel 4.18

  • For me, I still can't stand how zfs is integrated into Linux. That is one reason I don't use it. I also don't like how you can't add just one drive to a zfs array (I know this feature is coming).


    That IX blog is terrible. All it does to argue that zfs is better than btrfs is the fact that Redhat deprecated btrfs. Everything else is all about Freebsd not really openzfs and that is because zfs is the only filesystem Freenas has.


    I am still looking for a good CoW filesystem that is native to Linux. xfs is adding these features. bcachefs might happen. Personally, I have never had an issue with btrfs but I'm not using it for raid.

    omv 5.5.2 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Thanks for the reply. I have imported my pool but it is in a degraded state for some reason. It was only showing 2 out of the 6 drives. I realized I had unplugged cables. Plugged them in and only 5 were showing but it still imported. For whatever reason one drive is not showing up. Is it because I replaced my usb OS drive with an SSD? Maybe that device is taking the other in the device list? Or maybe I still have a cable unplugged.


    Anything I can check or look for in the logs? Ill try the cables again.

  • Check your cables again.

    Check your cables again.

    That was it...The one hard drive power cable wasn't in. Since changing to the SSD I had to pull power cables and add the cables (modular PSU) that would be enough for all the drives. That one was a tight fit against the case drive bay on the rear. I had overlooked it where it is located and didn't get a good look. Took it all off the shelf and laid on the floor. Had to import the pool again. But thankfully all the data was still there! Thanks for your help!

  • Frankly, I'm not a fan of ZFS. Out of the box, ZoL requires a handful of tweaks to "approximate" Linux permissions. (While that's another discussion entirely, my "introduction" to ZFS didn't go well when that issue became apparent.). However, in the search for something that would protect long term data stores; if one wants integrated protection from bitrot and other silent corruption, ZFS is the only viable option available. It's not even necessary to have a Z-array for protection - a basic volume would work, with copies=2 set, on sensitive filesystems.


    I'm not making more out of bitrot than it actually is. A flipped bit in a picture and a single pixel changes color. In a document an extraneous character may appear, a word may be spelled incorrectly, etc. However, the effects (cumulative) add up over the long haul which, I believe, should be given due consideration.


    For businesses of any size, I can't see where they could chose anything else (other than ZFS) at this point in time. Automated snapshots allow for the retrieval of business sensitive files, accidentally or deliberately deleted, for up to a year. And, snapshot retention can be adjusted for even longer periods. For that reason alone, ZFS should be supported for small and medium sized business use cases.
    __________________________________________


    BTRFS? It "sounds good" but what has been delivered is nowhere near living up to what has been promised. Because it's native to Linux, provides bitrot protection and has great (theoretical) features, I really "wanted" BTRFS for data storage. However, any objective assessment of the project leaves one with the idea that it's going to be a long time before the issues are worked out, if ever. (After being on the mailing list for some months, in times past, it seemed as if the clean up of the issues plaguingBTRFS was going nowhere.) In any case, BTRFS wouldn't be the first file system to be abandoned due to development delays and the loss of interest during/after an extended development period.

    So, IMHO, Ubuntu and Debian are the only choices for a decent solid server OS. OMV 5 must have ZFS. Ubuntu makes this easy and would eradicate a lot of the issues we see installing / updating Debian and keeping ZFS ticking. It may well be the better choice.

    Since ZFS is the only proven and mature CoW out there, at this point, I tend to agree with you. (Despite the PITA the kernel headers represent.) And it's not as if I "like" ZFS, now that I'm familiar with it. It's because ZFS is the only stable CoW option available. There's no other choice. So, if Ubuntu has a reasonable work around, that gets past the ZFS licensing issues and provides desirable repo features for @ryecoaaron and OMV's Dev's, using Ubuntu LTS as the base seems to make sense.


    The backports repo is quite a bit different especially where the kernel is involved. The LTS releases truly are supported for five years where Debian falls off to a subset of the packages later on. Ubuntu seems to put newer versions of packages in the LTS releases while maintaining the same stability as Debian as well. And if you want commercial support, Canonical's support is quite good (many times better than Redhat).

    While the decision is not mine to make, if it were, the above would be enough for me. Setting the ZFS issue aside - long term support and stability, for server development and maintenance, are not really choices. I'd see them as necessities.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!