Is ZFS supported in Kernel 4.13-4.15?

    • OMV 4.x
    • Resolved
    • Upgrade 3.x -> 4.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Just to add to the confusion. I have two setups, one a test vm that has been updated from the first 4.0 was available. This one has 4.9 kernel as the latest update available. This would have had the 4.0.7 omv update run on it.

      The other is a machine that has run omv for many years. I decided to update it from omv 2.x to 3 or 4 and use zfs. Upgrade command to 3 ran fine but there were some problems. Decided to see what upgrading to 4 would do for fum. Still had some issues so I did a clean install of 3.0 86 and upgraded that to 4. This one updated to the 4.13 kernel.

      Shouldn't the 4.0.7 update been included in the 4.0.9 update? Or is there some other way to fix it so I can use zfs?

      PS: Shouldn't this be in the bugtracker?
      If you make it idiot proof, somebody will build a better idiot.

      The post was edited 1 time, last by donh ().

    • donh wrote:

      Shouldn't the 4.0.7 update been included in the 4.0.9 update?
      Yes but it doesn't execute omv-mkconf apt. Plenty of things do that in the web interface including omv-extras now. If a person has the backports default that post above set to no, you will never get it.

      Krisbee wrote:

      But for future reference, when upgrading from latest(last?) OMV3 to OMV4 do I need to enable both pre-release updates and community-maintained updates along with testing plugin repos etc.?
      I never enable the pre-release updates or community maintained updates.

      donh wrote:

      Or is there some other way to fix it so I can use zfs?
      Why can't you use zfs? Because you have the 4.13 kernel?

      donh wrote:

      Shouldn't this be in the bugtracker?
      I didn't submit a bugtracker because I submitted a pull request to the github repo.

      Personally, I don't think a lot of the problems are valid because OMV 4.x was adopted too early. Most of them are easily fixable from the command line.
      omv 4.1.9 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.9
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Ok I found out what is pushing the 4.13 key. I installed the 3.86 iso update to latest 3.x. Then omv-release-upgraded to 4.x. Updated to the 4.0.9 and still no 4.13 kernel offered.

      I then did wget http://omv-extras.org/openmediavault-omvextrasorg_latest_all4.deb and dpkg -i openmediavault-omvextrasorg_latest_all4.deb. After that

      Source Code

      1. apt-get -s upgrade Reading package lists... Done
      2. Building dependency tree
      3. Reading state information... Done
      4. You might want to run 'apt --fix-broken install' to correct these.
      5. The following packages have unmet dependencies:
      6. openmediavault-omvextrasorg : Depends: dirmngr but it is not installed
      7. E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

      After installing dirmngr from web ui and checking for updates the 4.13 kernel shows up.



      FYI
      If you make it idiot proof, somebody will build a better idiot.

      The post was edited 2 times, last by donh: formating sucks ().

    • donh wrote:

      After installing dirmngr from web ui and checking for updates the 4.13 kernel shows up.
      apt-get -f install should have fixed that.
      omv 4.1.9 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.9
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      donh wrote:

      After installing dirmngr from web ui and checking for updates the 4.13 kernel shows up.
      apt-get -f install should have fixed that.
      Ether apt-get -f install or install from web ui will get past that. Installing openmediavault-zfs from the web ui causes errors. apt-get -f install shows this

      Source Code

      1. root@omv4zfs:~# apt-get -f install
      2. Reading package lists... Done
      3. Building dependency tree
      4. Reading state information... Done
      5. The following package was automatically installed and is no longer required:
      6. linux-image-4.9.0-0.bpo.4-amd64
      7. Use 'apt autoremove' to remove it.
      8. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
      9. 1 not fully installed or removed.
      10. After this operation, 0 B of additional disk space will be used.
      11. Setting up openmediavault-zfs (4.0) ...
      12. modprobe: FATAL: Module zfs not found in directory /lib/modules/4.13.0-0.bpo.1-amd64
      13. dpkg: error processing package openmediavault-zfs (--configure):
      14. subprocess installed post-installation script returned error exit status 1
      15. Processing triggers for openmediavault (4.0.9-1) ...
      16. Restarting engine daemon ...
      17. Errors were encountered while processing:
      18. openmediavault-zfs
      19. Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f2f6e533730>
      20. Traceback (most recent call last):
      21. File "/usr/lib/python3.5/weakref.py", line 117, in remove
      22. TypeError: 'NoneType' object is not callable
      23. Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f2f6e533730>
      24. Traceback (most recent call last):
      25. File "/usr/lib/python3.5/weakref.py", line 117, in remove
      26. TypeError: 'NoneType' object is not callable
      27. E: Sub-process /usr/bin/dpkg returned an error code (1)
      Display All
      I am guessing as above it's due to the 4.13 kernel.

      FYI
      If you make it idiot proof, somebody will build a better idiot.
    • donh wrote:

      I am guessing as above it's due to the 4.13 kernel.
      Yep, you probably don't have the linux headers installed for the 4.13 kernel and the zfs module included with stretch won't compile on 4.13.
      omv 4.1.9 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.9
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Skaronator wrote:

      donh wrote:

      The question should be why does omv-extras force the upgrade to 4.13 kernel and can that be fixed?
      OMV doesn't force the 4.13 kernel. Debian Backports does. Just disable the Backports if you don't want them.
      Of course OMV forces the kernel 4.13 due to its convoluted pinning under /etc/apt/ meaning it tracks backport kernels not stable kernels. Doesn't appear that simple as just disabling backports (which and what changes should be made?). The reason the zfs plugin pulls kernel 4.13 is the pinning for omv-extras:

      Source Code

      1. root@omv-vm:/etc/apt# cat ./preferences.d/omv-extras-org
      2. Package: virtualbox*
      3. Pin: release a=stretch-backports
      4. Pin-Priority: 500
      5. Package: zfs*
      6. Pin: release a=stretch-backports
      7. Pin-Priority: 500
      8. Package: spl-dkms
      9. Pin: release a=stretch-backports
      10. Pin-Priority: 500
      11. Package: libnvpair1linux
      12. Pin: release a=stretch-backports
      13. Pin-Priority: 500
      14. Package: libzfs2linux
      15. Pin: release a=stretch-backports
      16. Pin-Priority: 500
      17. Package: libzpool2linux
      18. Pin: release a=stretch-backports
      19. Pin-Priority: 500
      20. Package: libuutil1linux
      21. Pin: release a=stretch-backports
      22. Pin-Priority: 500
      23. Package: *
      24. Pin: release n=arrakis, origin dl.bintray.com
      25. Pin-Priority: 995
      26. Package: *
      27. Pin: release n=arrakis-testing, origin dl.bintray.com
      28. Pin-Priority: 996
      29. Package: *
      30. Pin: release n=arrakis-docker, origin dl.bintray.com
      31. Pin-Priority: 996
      32. Package: *
      33. Pin: release o=download.docker.com
      34. Pin-Priority: 1001
      Display All

      Also, don't forget as of debian 9, zfs modules are in contrib, but the default sources.list for omv does not have contrib or non-free:

      Source Code

      1. root@omv-vm:/etc/apt# cat sources.list
      2. #
      3. # deb cdrom:[Debian GNU/Linux 8 _Jessie_ - Official Snapshot amd64 LIVE/INSTALL Binary 20161229-12:48]/ stretch contrib main non-free
      4. #deb cdrom:[Debian GNU/Linux 8 _Jessie_ - Official Snapshot amd64 LIVE/INSTALL Binary 20161229-12:48]/ stretch contrib main non-free
      5. deb http://ftp.uk.debian.org/debian/ stretch main
      6. deb-src http://ftp.uk.debian.org/debian/ stretch main
      7. deb http://security.debian.org/ stretch/updates main contrib non-free
      8. deb-src http://security.debian.org/ stretch/updates main contrib non-free
      9. # stretch-updates, previously known as 'volatile'
      10. deb http://ftp.uk.debian.org/debian/ stretch-updates main contrib non-free
      11. deb-src http://ftp.uk.debian.org/debian/ stretch-updates main contrib non-free
      Display All
    • donh wrote:

      The question should be why does omv-extras force the upgrade to 4.13 kernel and can that be fixed?

      Krisbee wrote:

      The reason the zfs plugin pulls kernel 4.13 is the pinning for omv-extras:
      We can't win... People bitch if the kernel is too old and now people bitch if the kernel is too new. Just remember you are using an unreleased version... Here are the actual facts:

      1 - omv-extras does not enable the backports kernel. It does pin the zfs packages IN CASE the backports kernel is installed. Otherwise, the backports zfs packages which are required for the backports kernel (yes, I realize they don't work with the 4.13 kernel YET) would not be installed when the backports kernel is installed. These pinnings do NOTHING if the backports repo isn't enabled and it isn't enabled unless the next point applies.

      2 - OMV 3.x and 4.x by default will enable the backports repo and pin the kernel because of this line. The fact does not exist in the defaults file therefore defaults to yes. If you set OMV_APT_USE_KERNEL_BACKPORTS="no" in /etc/default/openmediavault and execute omv-mkconf apt, the backports kernel and zfs packages will not be installed (no, it won't uninstall them if they are already installed).


      Krisbee wrote:

      Also, don't forget as of debian 9, zfs modules are in contrib, but the default sources.list for omv does not have contrib or non-free:
      Where? I see them in wheezy, sid, and experimental but not stretch or even buster. Package list

      On an OMV 4.x system with backports enabled:
      $ apt-cache search zfs-modules
      zfs-dkms - OpenZFS filesystem kernel modules for Linux
      omv 4.1.9 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.9
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • @ryecoarron

      Not bitching, just trying to answer @donh question and understand what is happening, for my sake and possibly others. And of course, I appreciate this is an unreleased version.

      The package list you pointed to all ends in contrib in red. The point I was making was if you decided to disable stretch-backports in OMV4, thinking it might be a workaround to getting the zfs plugin to work, or thought you'd just install the required zfs packages at the CLI, apt-get in OMV4 doesn't find the stretch stable versions. You have to edit OMV4's default sources.list file.

      No one should object to having the latest kernel if you can easily revert to an earlier working kernel, which the WebUI provides for, should problems arise. Having a plugin that always pulls the latest kernel and fails to work is another, but related, issue.

      I realise now, that my answer to the OP's question should simply have been: "The zfs modules do not build against kernel 4.13 because of a bug. Check dkms status to ensure you still have zfs modules built for previous kernel. Set default boot kernel in WebUI to previous kernel. No need to change anything else, but don't update kernel again until you know problem has been fixed"

      Of course, this is not an answer to someone who has recently updated to OMV4 and just decided that they would like to use zfs.
    • Krisbee wrote:

      The package list you pointed to all ends in contrib in red. The point I was making was if you decided to disable stretch-backports in OMV4, thinking it might be a workaround to getting the zfs plugin to work, or thought you'd just install the required zfs packages at the CLI, apt-get in OMV4 doesn't find the stretch stable versions. You have to edit OMV4's default sources.list file.
      We are talking about two different things. I realize that the zfs stuff is in contrib. I was pointing out that there are no zfs modules in stretch or buster even in contrib. As for contrib not being in sources.list, I guess we will have to look at that.

      Krisbee wrote:

      Having a plugin that always pulls the latest kernel and fails to work is another, but related, issue.
      No plugin pulls the latest kernel. If you have backports enabled and install zfs, sure you will get the latest kernel but why do you have backports enabled then? If you have a backports kernel installed (say 4.12), installing the zfs plugin shouldn't install 4.13. And the failing will stop once the backports kernel stabilizes at a version.

      Krisbee wrote:

      Not bitching, just trying to answer @donh question and understand what is happening, for my sake and possibly others. And of course, I appreciate this is an unreleased version.
      This is still a lot of posts to "discuss" how something unreleased works.
      omv 4.1.9 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.9
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Krisbee wrote:

      Having a plugin that always pulls the latest kernel and fails to work is another, but related, issue.
      No plugin pulls the latest kernel. If you have backports enabled and install zfs, sure you will get the latest kernel but why do you have backports enabled then? If you have a backports kernel installed (say 4.12), installing the zfs plugin shouldn't install 4.13. And the failing will stop once the backports kernel stabilizes at a version.


      Yes, that does seems logical, but it's not exactly what is happening. Here is my test:

      1. Fresh install from 3.0.86 iso fully upated to OMV 3.0.91, with jessie-backports but no plugins installed.
      2. Upgrade from OMV 3.0.91 to OMV 4.0.9 using "omv-release upgrade"
      3. After reboot to OMV4, kernel is already from stretch-backports:

      Source Code

      1. root@omv-vm:~# uname -a
      2. Linux omv-vm 4.9.0-0.bpo.4-amd64 #1 SMP Debian 4.9.51-1~bpo8+1 (2017-10-17) x86_64 GNU/Linux
      3. root@omv-vm:~# dpkg -l | grep linux-
      4. ii firmware-linux-free 3.4 all Binary firmware for various drivers in the Linux kernel ii firmware-linux-nonfree 20161130-3~bpo8+1 all Binary firmware for various drivers in the Linux kernel (meta-package)
      5. ii linux-base 4.3~bpo8+1 all Linux image base package
      6. ii linux-image-4.9.0-0.bpo.3-amd64 4.9.30-2+deb9u5~bpo8+1 amd64 Linux 4.9 for 64-bit PCs
      7. ii linux-image-4.9.0-0.bpo.4-amd64 4.9.51-1~bpo8+1 amd64 Linux 4.9 for 64-bit PCs
      8. ii linux-image-amd64 4.9+80+deb9u2~bpo8+1 amd64 Linux for 64-bit PCs (meta-package)
      9. root@omv-vm:~# dkms status
      10. -bash: dkms: command not found
      11. root@omv-vm:~#
      12. root@omv-vm:~# apt-cache policy linux-image-amd64
      13. linux-image-amd64:
      14. Installed: 4.9+80+deb9u2~bpo8+1
      15. Candidate: 4.9+80+deb9u2
      16. Version table:
      17. 4.9+80+deb9u2 500
      18. 500 http://ftp.uk.debian.org/debian stretch/main amd64 Packages
      19. *** 4.9+80+deb9u2~bpo8+1 500
      20. 100 http://httpredir.debian.org/debian jessie-backports/main amd64 Packages
      21. 100 /var/lib/dpkg/status
      Display All

      4. Still had to use omv-mkconf apt / apt-get update get correct openmediavault-kernel-backports.list

      Source Code

      1. root@omv-vm:/# apt-cache policy linux-image-amd64
      2. linux-image-amd64:
      3. Installed: 4.9+80+deb9u2~bpo8+1
      4. Candidate: 4.13+86~bpo9+1
      5. Version table:
      6. 4.13+86~bpo9+1 500
      7. 100 http://httpredir.debian.org/debian stretch-backports/main amd64 Packages
      8. 4.9+80+deb9u2 500
      9. 500 http://ftp.uk.debian.org/debian stretch/main amd64 Packages
      10. *** 4.9+80+deb9u2~bpo8+1 100
      11. 100 /var/lib/dpkg/status
      Display All

      5. Uploaded and installed omv4 extras

      6. Check package status:


      Source Code

      1. root@omv-vm:~# dpkg -l | grep openmed
      2. ii openmediavault 4.0.9-1 all openmediavault - The open network attached storage solution
      3. ii openmediavault-keyring 1.0 all GnuPG archive keys of the OpenMediaVault archive
      4. ii openmediavault-omvextrasorg 4.1.0 all OMV-Extras.org Package Repositories for OpenMediaVault
      5. root@omv-vm:~# dpkg -l | grep -Ew "zfs|spl"
      6. root@omv-vm:~#


      7. No system package upgrade done yet. NOTE: no kernel headers are installed on system by default at this stage,

      8. Attempt to install zfs plugin via webUI. Zfs plugin install fails, error messages show plugin has installed later backport kernel 4.13 and headers, spl/zfs module build has failed.

      9. Post fail of zfs plugin package check:


      Source Code

      1. root@omv-vm:~# dpkg -l | grep linux-
      2. ii firmware-linux-free 3.4 all Binary firmware for various drivers in the Linux kernel
      3. ii firmware-linux-nonfree 20161130-3~bpo8+1 all Binary firmware for various drivers in the Linux kernel (meta-package)
      4. ii linux-base 4.3~bpo8+1 all Linux image base package
      5. ii linux-compiler-gcc-6-x86 4.9.51-1 amd64 Compiler for Linux on x86 (meta-package)
      6. iF linux-headers-4.13.0-0.bpo.1-amd64 4.13.4-2~bpo9+1 amd64 Header files for Linux 4.13.0-0.bpo.1-amd64
      7. ii linux-headers-4.13.0-0.bpo.1-common 4.13.4-2~bpo9+1 all Common header files for Linux 4.13.0-0.bpo.1
      8. iU linux-headers-amd64 4.13+86~bpo9+1 amd64 Header files for Linux amd64 configuration (meta-package)
      9. ii linux-image-4.9.0-0.bpo.3-amd64 4.9.30-2+deb9u5~bpo8+1 amd64 Linux 4.9 for 64-bit PCs
      10. ii linux-image-4.9.0-0.bpo.4-amd64 4.9.51-1~bpo8+1 amd64 Linux 4.9 for 64-bit PCs
      11. ii linux-image-amd64 4.9+80+deb9u2~bpo8+1 amd64 Linux for 64-bit PCs (meta-package)
      12. ii linux-kbuild-4.13 4.13.4-2~bpo9+1 amd64 Kbuild infrastructure for Linux 4.13
      13. ii linux-libc-dev:amd64 4.9.51-1 amd64 Linux support headers for userspace development
      14. root@omv-vm:~# dpkg -l | grep -Ew "zfs|spl"
      15. iU openmediavault-zfs 4.0 amd64 OpenMediaVault plugin for ZFS
      16. ii spl-dkms 0.6.5.11-1~bpo9+1 all Solaris Porting Layer kernel modules for Linux
      17. ii zfs-dkms 0.6.5.11-1~bpo9+1 all OpenZFS filesystem kernel modules for Linux
      18. iU zfs-zed 0.6.5.11-1~bpo9+1 amd64 OpenZFS Event Daemon
      19. root@omv-vm:~# dkms status
      20. spl, 0.6.5.11: added
      21. zfs, 0.6.5.11: added
      22. root@omv-vm:~# zpool status
      23. The ZFS modules are not loaded.
      24. Try running '/sbin/modprobe zfs' as root to load them.
      25. root@omv-vm:~# dpkg -l | grep openmed
      26. ii openmediavault 4.0.9-1 all openmediavault - The open network attached storage solution
      27. ii openmediavault-keyring 1.0 all GnuPG archive keys of the OpenMediaVault archive
      28. ii openmediavault-omvextrasorg 4.1.0 all OMV-Extras.org Package Repositories for OpenMediaVault
      29. iU openmediavault-zfs 4.0 amd64 OpenMediaVault plugin for ZFS
      30. root@omv-vm:~#
      Display All


      openmediavault-zfs failed to install properly, but has added packages to system and is itself shown as installed.

      So despite there already being an active backport kernel on my OMV4 test install, zfs pluign has installed the latest 4.13 kernel because active kernel was from jessie-backports not stretch-backports. I thought if you did an upgrade from jessie to stretch on a straight Debian install, you would end up with a new stable stretch kernel. So does a OMV release upgrade do things differently?

      Output from zfs plugin is here: pastebin.com/mLrtnJqm

      The post was edited 1 time, last by Krisbee ().

    • First off I know this is a testing work in progress. I have been testing since very early versions. Testing is to provide feedback for the future, not bitch. Don't take it personally. We are trying to contribute also.

      As of two days ago, a clean 3.86 and upgrading to 4.0.9 works fine. All updated you are still at 4.9 kernel. Installing omv-extras at cli goes fine. Checking for updates shows one update. dirmngr After that is updated without any extras installed, 4.13 kernel is an available update. Adding the zfs pluggin will pull in the 4.13 kernel.


      Could there be a switch to not change to a new kernel version unless checked? Unfortunately I don't know what else that might break.


      Thanks
      If you make it idiot proof, somebody will build a better idiot.
    • donh wrote:

      Could there be a switch to not change to a new kernel version unless checked? Unfortunately I don't know what else that might break.
      You asking for a change in apt/dpkg behavior which we cannot change.

      donh wrote:

      Testing is to provide feedback for the future, not bitch. Don't take it personally. We are trying to contribute also.
      I get it and I'm not taking it personally and I think you should contribute. The problem is that this is a lot of time spent on a temporary problem and the feedback just seems a little upset that zfs is broken because Debian not OMV released a new kernel but didn't update the zfs packages to work with it. And no one is reading what I am posting. Please read what I post next very carefully::


      The 4.13 kernel is being installed because it is the newest kernel. The 4.9 kernel you have installed when you update from OMV 3.x to 4.x is actually the 4.9 Jessie Backports kernel not the 4.9 Stretch kernel. Since omv-release-upgrade doesn't execute omv-mkconf apt, the backports repo file stays at jessie-backports and isn't updated to stretch-backports. This makes apt not upgrade the kernel to the 4.9 stretch kernel because they are same version. When you install omv-extras, it does execute omv-mkconf apt which enables the stretch-backports repo. This will make it show up in the updates list but it will not be installed unless you tell it to. BUT if you install the zfs plugin, it has a dependency on linux-headers-amd64. This is a meta package that points to the latest linux-headers which obviously need the matching kernel. So, it installs the 4.13 kernel destroying the whole world. If you upgraded a 3.x install with the linux-headers-amd64 package from 3.x installed (which wasn't upgraded during omv-release-upgrade because it matched the available version in stretch and stretch-backports wasn't enable to get the latest stretch-backports version), then the correct headers won't be installed even if you manually installed the 4.12 kernel causing the zfs plugin to break again.

      SO, if you want a OMV 4.x install with working zfs, set the OMV_APT_USE_KERNEL_BACKPORTS="no" in /etc/default/openmediavault before upgrading to 4.x. Then you will keep the 4.9 kernel and never ever install the 4.13 kernel. When you install OMV extras and zfs plugin, the 4.9 linux headers will be installed making zfs module compiling work.
      omv 4.1.9 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.9
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!