kernel plugin: OMV6 to OMV7 upgrade followed by pve kernel install creates incorrect zfs config

    • Official Post

    this clearly wrong.

    Why is it wrong? The kernel plugin doesn't remove zfs-dkms after installing the proxmox kernel or removing the debian kernels. zfs-dkms being installed with only the proxmox kernels doesn't actually build the module.


    I'm not sure what would change with the upgrade script because apt is just doing what the control file says. On amd64, zfs-dkms will only be installed if the pve-headers package is not installed.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • ryecoaaron The title of my OP is pointing blame at the wrong thing. It's the ovm-release-upgrade script that can lead to unwanted result. The pvekernel.list is dropped early on in the process, when new versions of the kernel and zfs plugins are installed there's nothing to trigger the installation of pve kernel 6.5 and the upgrade script drags back in dkms, the deb versions of zfs pacakges etc and builds zfs modules against the current bpo11 pve 6.2 kernel. On re-boot, the end result is definitely not an upgrade of pve kernel and pve zfs.


    Starting with this:


    Ends up with this, after re-boot:


    • Official Post

    The pvekernel.list is dropped early on in the process, when new versions of the kernel and zfs plugins are installed there's nothing to trigger the installation of pve kernel 6.5 and the upgrade script drags back in dkms

    pvekernel.list is removed on purpose - https://github.com/OpenMediaVa…mvextras-sources-list#L38


    The upgrade should not install the 6.5 kernel for two reasons.

    1. Someone may not want the 6.5 kernel. The kernel plugin can install the 6.2 or 6.5 kernel on OMV 7.x.
    2. The proxmox kernel installed from OMV 6.x should work just fine on OMV 7.x and bookworm. I didn't want the upgrade process to have to deal with proxmox kernel as well. My plan was for people to manually install the kernel after a successful upgrade using the kernel plugin.


    The script doesn't install dkms. The only thing that should install that are the dependencies in the zfs plugin BUT it should not install zfs-dkms if pve-headers are installed - https://github.com/OpenMediaVa…master/debian/control#L16


    I will have to try an upgrade to see what exactly is installing dkms/zfs-dkms. I would *guess* that it is caused by zfs modules being newer in Deb12 than what is currently installed from Deb11/OMV 6.x before the upgrade. This isn't fun to fix since there are so many combos of kernels and architectures that have to be thought of.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • ryecoaaron Yes, I know the pvekernel.list is removed during the omv-release-upgrade in one of the run-parts. If your plan was for people to manually install 6.2/6.5 post update, has that ever been stated anywhere? If dkms/zfs-dkms is installed as a consequence of running the upgrade script, then in my book it's the script that does it. I did look at the openmediavualt-zfs deb package control, so quite possibly its because Debian 12 packages are newer than pve 2.1.14 , OMV's apt pinning prefs etc. are not straightforward. I can quite see how this is a tricky problem. In the meantime, I believe my simple minded workaround in #29 above overcomes the problem (although you still get a wasteful dkms build which is subsequently removed)


    Ironically the upgrade from OMV6 to OMV7 is fine if you start with a deb11 kernel along with zfs. But even here there is an anomaly which perhaps you can explain. Starting with a deb11 kernel and zfs, the upgrade leaves you with zfs version zfs-2.2.3. If you install OMV7 from scratch and install zfs when using a deb12 kernel, you get zfs 2.1.11 even when backports are installed on the system. Why does that happen?

    • Official Post

    If your plan was for people to manually install 6.2/6.5 post update, has that ever been stated anywhere?

    I didn't really have a plan. I just had the script remove the repo because I wanted the kernel plugin to setup the proper repo. I just figured most people would have the 6.2 kernel installed already and the system would ok with it after.


    There could be a mix of the following:

    • any combination four kernels installed - debian regular, debian backports, pve 5.19, pve 6.2
    • backports enabled or not
    • pve repo enabled or not
    • zfs installed or not
    • zfs-dkms installed or not.

    If someone can fix the upgrade script to accommodate all of those scenarios, I welcome the PR. Until then, I still haven't tried anything.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • ryecoaaron Users who need DKMS for drivers is another scenario you could add to the list. Unfortunately it looks like the PVE kernel is the fly in the ointment. In my tests, the PVE repo was never replaced during the upgrade.


    Rather than trying to fix the omv-release-upgrade script per se, would applying a post-upgrade fix-script after re-booting into OMV7 be a viable approach for those starting from OMV6 + PVE + ZFS?

    • Official Post

    would applying a post-upgrade fix-script after re-booting into OMV7 be a viable approach for those starting from OMV6 + PVE + ZFS?

    Sure. Would the user run this script? Who is going to write it?

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post

    The problem is that backports is most likely enabled and backports has a newer version of zfs-zed and zfsutils-linux than what is in the proxmox kernel. When these debian packages are installed, the depend on zfs-dkms.


    So, people could disable backports before upgrading (I didn't test that) OR you can fix it after the upgrade with:


    omv-changebackports no

    apt-get install zfs-zed=2.2.2-pve2 zfsutils-linux=2.2.2-pve2

    apt-get purge zfs-dkms dkms


    No need to remove the plugin.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post

    This problem will be easier once the proxmox repo has a newer version of zfs.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Your post-upgrade fix fails as there is no pve repo post update. The OMV6/7 upgrade deletes the pve repo and AFAIK the only thing that installs a pve repo on OMV is when omv-installproxmox is run.


    The far better option is to disable backports before upgrading OMV6 to OMV7. This streamlines the upgrade, dksm/zfs-dksm are not re-installed and no wasteful dlkms zfs modules build take place.


    So in OMV6 at the CLI:


    1. omv-changebackports no

    2., omv-release-upgrade

    3. re-boot


    After the first re-boot into OMV7 you will still need to


    1. Install pve kernel 6.2 or 6.5 in order that the pve repo is installed.

    2. re-boot into chosen pve kernel.

    3. Update OMV7 in the normal way via WebUI Update Management.

    4. Via the WebUI System omv-extras enable backports.

    • Official Post

    Your post-upgrade fix fails as there is no pve repo post update. The OMV6/7 upgrade deletes the pve repo and AFAIK the only thing that installs a pve repo on OMV is when omv-installproxmox is run.

    I installed the 6.5 kernel before doing that. I guess that step was assumed no matter what the solution. I guess I could make omv-release-upgrade stop deleting it but my energy to test it is very low.


    4. Via the WebUI System omv-extras enable backports.

    If the packages are newer in the Debian backports repo, this could still lead to the zfs packages being updated and dkms/zfs-dkms being installed. I could have something (zfs plugin or kernel plugin) create an apt preferences file but it would be very confusing due to the number of options possible. I think the safest thing to do is leave backports disabled. I don't think it is needed for anything on OMV 7.x currently anyway.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • AFAIU, the default in OMV7 is OMV_APT_USE_KERNEL_BACKPORTS=YES. Happy to leave if off. The procedure I outlined in #52 also works when a user has left debian kernels on their system and used dkms prior to installing a pve kenrel in OMV6. for example:


    OMV6 start condition:


    OMV7 end condition:

    • Official Post

    AFAIU, the default in OMV7 is OMV_APT_USE_KERNEL_BACKPORTS=YES. Happy to leave if off

    That is correct. It just isn't a good idea when backports packages are newer than proxmox.

    omv 7.7.9-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0.2 | kvm 7.1.8 | compose 7.6.9 | cterm 7.8.7 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.2


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Took the plunge on upgrading my live server, after clonezilla-ing it.

    1. Ran sudo omv-upgrade ok.

    2. Ran sudo omv-release-upgrade which returned an error again:

    Code
     /tmp/apt-dpkg-install-YdKkJs/012-firmware-ath9k-htc_1.4.0-108-gd856466+dfsg1-1.3+deb12u1_all.deb
    E: Sub-process /usr/bin/dpkg returned an error code (1)

    3. Ran sudo apt --fix-broken install, and took the options to install the maintainer's version of config files. Update-initramfs errored with "Operation was interrupted before it could finish".

    4. Ran fix6to7upgrade with no errors.

    5. Rebooted ok. Shares available, web gui available etc. All seems ok at first glance.


    The only thing out of whack is that zfs -V returns:

    zfs-2.2.3-pve1
    zfs-kmod-2.1.9-pve1


    And uname -r returns:

    6.1.15-1-pve


    EDIT: Upgraded to Proxmox 6.5 via web gui and zfs -V now shows zfs-2.2.3-pve1 & zfs-kmod-2.2.3-pve1

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!