PVE Kernel removed going from 6 to 7 and can't reinstall

Krisbee · 13. März 2024

I just seen the output for OMV6. After a pve kernel is installed in OMv6 there is no reason to keep debian kernels on the same system. The whole point is to avoid potential problems with dkms building zfs modules. Installing a pve kernel removes debian zfs packages and replaces then with its own, along with pve firmware, etc. Once you move to a pve kernel, there is no mechanism to downgrade ( see: .https://forum.openmediavault.org/index.php?thread/50865-stock-kernel-with-zfs/&postID=377386#post377386 )

In order to clean up your OMV6, do the following:

1. remove all "non-proxmox" kernels via the WebUI "System | Kernel" tab.

2. check dkms status, there should be no entries for zfs 2.1.11. If any remain, remove them with "dkms remove zfs/2.1.11 --all"

3. If necessary, remove " veeamsnap, 6.1.0.1498, 6.1.0-0.deb11.13-amd64" from dkms.

Your dpkg outputs show you have unnecessary packages and/or config files ( indicated by "rc" in first column) cluttering your system and you should remove these. You can purge all the old pve kernel config files left behind when old pve kernel packages have been removed, e.g:

4. apt purge pve-kernel-5.15.12-1-pve pve-kernel-5.15.17-1-pve ...

5. Using apt autoremove will show you which other packages can be removed. Just doublecheck the list before accepting.

You can re-post your outputs after this

Krisbee · 13. März 2024

spupuz For the moment, could you please refrain from posting in this thread. I've already pointed out the thread which includes your config in #9 above.

Zitat von Krisbee

Thanks for that. The output suggest you've run into the kind of problem I posted about here: kernel plugin: OMV6 to OMV7 upgrade followed by pve kernel install creates incorrect zfs config

spupuz · 13. März 2024

plugin openmediavault-zfs require zfs-dkms this is the problem, this is why happens to have installed the dkms modules.

Krisbee · 13. März 2024

Zitat von spupuz

plugin openmediavault-zfs require zfs-dkms this is the problem, to this is why happens to have installed the dkms modules.

Not true in all cases. I ask again that your refrain for posting in this thread at the moment. We're not discussing how/why the OP's OMV6 arrived at its current state, but how to clean it up prior to a possible upgrade. Please start a new thread to discuss such issues if you wish.

spupuz · 13. März 2024

Zitat von Krisbee

Not true in all cases. I ask again that your refrain for posting in this thread at the moment. We're not discussing how/why the OP's OMV6 arrived at its current state, but how to clean it up prior to a possible upgrade. Please start a new thread to discuss such issues if you wish.

right just tested, but why then got installed during the upgrade the zfs-dfkms?
i'm just trying to understand since i have the same configuration of him and i also want to have a clean upgrade path so please be kind

Krisbee · 13. März 2024

Zitat von spupuz

right just tested, but why then got installed during the upgrade the zfs-dfkms?
i'm just trying to understand since i have the same configuration of him and i also want to have a clean upgrade path so please be kind

As stated elsewhere the problem seems to be in the omv-release-upgrade script and the interaction with the kernel/zfs plugins. This is for votdev and/or @ryecoarran to look at.

marc_al · 13. März 2024

Hello,

Thanks for the help. Here is what I have now

Code

uname -a
Linux openmediavault 5.15.143-1-pve #1 SMP PVE 5.15.143-1 (2024-02-08T18:12Z) x86_64 GNU/Linux

Code

zfs -V
zfs-2.1.14-pve1
zfs-kmod-2.1.14-pve1

Code

sudo dkms status
veeamsnap, 6.1.0.1498, 5.15.143-1-pve, x86_64: installed

Code

dpkg -l | grep zfs
ii  libvirt-daemon-driver-storage-zfs    8.0.0-1~bpo11+1                             amd64        Virtualization daemon ZFS storage driver
rc  libzfs2                              0.6.5.7-8-wheezy                            amd64        Native ZFS filesystem library for Linux
ii  libzfs4linux                         2.1.14-pve1                                 amd64        OpenZFS filesystem library for Linux - general support
ii  openmediavault-zfs                   6.0.14                                      amd64        OpenMediaVault plugin for ZFS
rc  zfs-auto-snapshot                    1.2.4-2                                     all          ZFS automatic snapshot service
ii  zfs-dkms                             2.1.11-1~bpo11+1                            all          OpenZFS filesystem kernel modules for Linux
ii  zfs-zed                              2.1.14-pve1                                 amd64        OpenZFS Event Daemon
rc  zfsutils                             0.6.5.7-8-wheezy                            amd64        command-line tools to manage ZFS filesystems
ii  zfsutils-linux                       2.1.14-pve1                                 amd64        command-line tools to manage OpenZFS filesystems

Code

dpkg -l | grep pve
ii  ifupdown                             0.8.36+pve2                                 amd64        high level tools to configure network interfaces
ii  liblvm2cmd2.02:amd64                 2.02.168-pve6                               amd64        LVM2 command library
ii  libnvpair3linux                      2.1.14-pve1                                 amd64        Solaris name-value library for Linux
ii  libuutil3linux                       2.1.14-pve1                                 amd64        Solaris userland utility library for Linux
ii  libzfs4linux                         2.1.14-pve1                                 amd64        OpenZFS filesystem library for Linux - general support
ii  libzpool4linux                       2.0.6-pve1~bpo10+1                          amd64        OpenZFS pool library for Linux
ii  libzpool5linux                       2.1.14-pve1                                 amd64        OpenZFS pool library for Linux
ii  pve-firmware                         3.6-6                                       all          Binary firmware code for the pve-kernel
ii  pve-headers                          7.4-1                                       all          Default Proxmox VE Kernel Headers
ii  pve-headers-5.15                     7.4-11                                      all          Latest Proxmox VE Kernel Headers
ii  pve-headers-5.15.143-1-pve           5.15.143-1                                  amd64        Proxmox Kernel Headers
ii  pve-kernel-5.15                      7.4-11                                      all          Latest Proxmox VE Kernel Image
ii  pve-kernel-5.15.143-1-pve            5.15.143-1                                  amd64        Proxmox Kernel Image
ii  smartmontools                        7.2-pve3                                    amd64        control and monitor storage systems using S.M.A.R.T.
ii  spl                                  2.1.14-pve1                                 all          Solaris Porting Layer user-space utilities for Linux (dummy)
ii  zfs-zed                              2.1.14-pve1                                 amd64        OpenZFS Event Daemon
ii  zfsutils-linux                       2.1.14-pve1                                 amd64        command-line tools to manage OpenZFS filesystems

Alles anzeigen

Code

dpkg -l | grep openm
ii  libopenmpt0:amd64                    0.4.11-1                                    amd64        module music library based on OpenMPT -- shared library
ii  openmediavault                       6.9.14-1                                    all          openmediavault - The open network attached storage solution
ii  openmediavault-backup                6.1.1                                       all          backup plugin for OpenMediaVault.
ii  openmediavault-cputemp               6.1.3                                       all          cpu temperature plugin for openmediavault
ii  openmediavault-flashmemory           6.2                                         all          folder2ram plugin for openmediavault
ii  openmediavault-kernel                6.4.9                                       all          kernel package
ii  openmediavault-keyring               1.0.2-2                                     all          GnuPG archive keys of the openmediavault archive
ii  openmediavault-kvm                   6.2.18                                      all          kvm plugin for OpenMediaVault.
ii  openmediavault-nut                   6.0.7-1                                     all          openmediavault Network UPS Tools (NUT) plugin
ii  openmediavault-omvextrasorg          6.3.6                                       all          OMV-Extras.org Package Repositories for OpenMediaVault
ii  openmediavault-sharerootfs           6.0.3-1                                     all          openmediavault share root filesystem plugin
ii  openmediavault-symlinks              6.0.4                                       all          OpenMediaVault symlinks plugin
ii  openmediavault-usbbackup             6.0.5-1                                     all          openmediavault USB/eSATA backup plugin
ii  openmediavault-zfs                   6.0.14                                      amd64        OpenMediaVault plugin for ZFS

Alles anzeigen

Code

dpkg -l | grep linux
ii  binutils-x86-64-linux-gnu            2.35.2-2                                    amd64        GNU binary utilities, for x86-64-linux-gnu target
ii  console-setup-linux                  1.205                                       all          Linux specific part of console-setup
ii  libnvpair3linux                      2.1.14-pve1                                 amd64        Solaris name-value library for Linux
ii  libselinux1:amd64                    3.1-3                                       amd64        SELinux runtime shared libraries
ii  libselinux1-dev:amd64                3.1-3                                       amd64        SELinux development headers
ii  libuutil3linux                       2.1.14-pve1                                 amd64        Solaris userland utility library for Linux
ii  libv4l-0:amd64                       1.20.0-2                                    amd64        Collection of video4linux support libraries
ii  libv4lconvert0:amd64                 1.20.0-2                                    amd64        Video4linux frame format conversion library
ii  libzfs4linux                         2.1.14-pve1                                 amd64        OpenZFS filesystem library for Linux - general support
ii  libzpool4linux                       2.0.6-pve1~bpo10+1                          amd64        OpenZFS pool library for Linux
ii  libzpool5linux                       2.1.14-pve1                                 amd64        OpenZFS pool library for Linux
ii  linux-base                           4.6                                         all          Linux image base package
ii  linux-compiler-gcc-4.8-x86           3.16.51-3+deb8u1                            amd64        Compiler for Linux on x86 (meta-package)
ii  linux-compiler-gcc-6-x86             4.19.67-2+deb10u2~bpo9+1                    amd64        Compiler for Linux on x86 (meta-package)
ii  linux-libc-dev:amd64                 5.10.209-2                                  amd64        Linux support headers for userspace development
ii  util-linux                           2.36.1-8+deb11u1                            amd64        miscellaneous system utilities
ii  zfsutils-linux                       2.1.14-pve1                                 amd64        command-line tools to manage OpenZFS filesystems

Alles anzeigen

Code

ls -l /etc/apt/sources.list.d
total 32
-rw-r--r-- 1 root root  115 Dec 31 06:48 omvdocker.list
-rw-r--r-- 1 root root  141 Dec 31 06:48 omvextras.list
-rw-r--r-- 1 root root  150 Aug  8  2023 openmediavault-kernel-backports.list
-rw-r--r-- 1 root root   60 Jun  8  2023 openmediavault-local.list
-rw-r--r-- 1 root root  336 Mar 13 08:46 openmediavault-os-security.list
-rw-r--r-- 1 root root 1462 Jun  8  2023 openmediavault.list
-rw-r--r-- 1 root root  144 Mar 13 10:14 pvekernel.list
-rw-r--r-- 1 root root   96 Jun  8  2023 veeam.list

Krisbee · 13. März 2024

marc_al OK, the clutter has mostly gone. You don't need any package that refers to wheezy left on your system, or that has bpo10 in the name. Nor do you need zfs-dkms 2.1.11. But removing it may also remove openmediavault-zfs, which means just installing it again afterwards. Just check this leaves you with correct zfs packages on OMV6..

vperas · 13. März 2024

Hey Kriss, it this otput OK for a ZFS updated OMV:

Code

root@nas:~# uname -a
Linux nas 6.5.13-1-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.13-1 (2024-02-05T13:50Z) x86_64 GNU/Linux

root@nas:~# zfs -V
zfs-2.2.2-pve2
zfs-kmod-2.2.2-pve1

root@nas:~# dkms status
-bash: dkms: command not found

root@nas:~# dpkg -l | grep zfs
ii  libzfs4linux                       2.2.2-pve2                          amd64        OpenZFS filesystem library for Linux - general support
ii  openmediavault-zfs                 7.0.5                               amd64        OpenMediaVault plugin for ZFS
ii  zfs-zed                            2.2.2-pve2                          amd64        OpenZFS Event Daemon
ii  zfsutils-linux                     2.2.2-pve2                          amd64        command-line tools to manage OpenZFS filesystems

Alles anzeigen

Soma · 13. März 2024

Zitat von Krisbee

marc_al OK, the clutter has mostly gone. You don't need any package that refers to wheezy left on your system, or that has bpo10 in the name. Nor do you need zfs-dkms 2.1.11. But removing it may also remove openmediavault-zfs, which means just installing it again afterwards. Just check this leaves you with correct zfs packages on OMV6..

Just a FYI (in case you forgot)

Adding --dry-run to any apt command will simulate the outcome.

That way is easy to see if it will damage any thing

Krisbee · 13. März 2024

Zitat von Soma

Just a FYI (in case you forgot)

Adding --dry-run to any apt command will simulate the outcome.
That way is easy to see if it will damage any thing

-s is simpler

marc_al · 13. März 2024

Hello,

I purge all the remaining files (and reboot) and I report back after the upgrade.

Thank again

Marc

marc_al · 13. März 2024

Zitat von Soma

Just a FYI (in case you forgot)

Adding --dry-run to any apt command will simulate the outcome.
That way is easy to see if it will damage any thing

Thank you for the information.

What would be great is to have snapshots of the omv drive so if something goes wrong there is no need to restore a backup

Soma · 13. März 2024

Zitat von marc_al

What would be great is to have snapshots of the omv drive so if something goes wrong there is no need to restore a backup

Not possible OOB.

In order to have that, you would need to do it by hand when installing DebianOS.

I did it on a whim, on a Pi4 some time ago and it worked flawless.

Create a BTRFS root partition with snapshots

Install and run SNAPPER on the root partition.

SNAPSHOTS were done on every boot/reboot && on apt run.

Not impossible but not for the faint of heart (these are for Raspberry but you might find/search for amd64)

Boot Btrfs root partition with Raid1 on Raspberri Pi | Ephestione's HQ

Raspbian and btrfs (feldspaten.org)

Raspbian Btrfs root filesystem guide (2017-06) - HackMD

Snapper - ArchWiki (archlinux.org)

Since Pis don't use GRUB, to boot from a previous snapshot was only a matter of changing the cmdline.txt to use a different subvol.

On grub, I have no idea

spupuz · 13. März 2024

Zitat von vperas

Hey Kriss, it this otput OK for a ZFS updated OMV:

Code

root@nas:~# uname -a
Linux nas 6.5.13-1-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.13-1 (2024-02-05T13:50Z) x86_64 GNU/Linux

root@nas:~# zfs -V
zfs-2.2.2-pve2
zfs-kmod-2.2.2-pve1

root@nas:~# dkms status
-bash: dkms: command not found

root@nas:~# dpkg -l | grep zfs
ii  libzfs4linux                       2.2.2-pve2                          amd64        OpenZFS filesystem library for Linux - general support
ii  openmediavault-zfs                 7.0.5                               amd64        OpenMediaVault plugin for ZFS
ii  zfs-zed                            2.2.2-pve2                          amd64        OpenZFS Event Daemon
ii  zfsutils-linux                     2.2.2-pve2                          amd64        command-line tools to manage OpenZFS filesystems

Alles anzeigen

in my test env i have this situation:

root@omvbatest:~# zfs -V

zfs-2.2.2-pve2

zfs-kmod-2.1.14-pve1

why are these different?

Krisbee · 13. März 2024

Zitat von vperas

Hey Kriss, it this otput OK for a ZFS updated OMV:

Code

root@nas:~# uname -a
Linux nas 6.5.13-1-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.13-1 (2024-02-05T13:50Z) x86_64 GNU/Linux

root@nas:~# zfs -V
zfs-2.2.2-pve2
zfs-kmod-2.2.2-pve1

root@nas:~# dkms status
-bash: dkms: command not found

root@nas:~# dpkg -l | grep zfs
ii  libzfs4linux                       2.2.2-pve2                          amd64        OpenZFS filesystem library for Linux - general support
ii  openmediavault-zfs                 7.0.5                               amd64        OpenMediaVault plugin for ZFS
ii  zfs-zed                            2.2.2-pve2                          amd64        OpenZFS Event Daemon
ii  zfsutils-linux                     2.2.2-pve2                          amd64        command-line tools to manage OpenZFS filesystems

Alles anzeigen

Something still amiss in the output "zfs -V ". The module number does not match the utils number.

spupuz · 13. März 2024

Zitat von Krisbee

Something still amiss in the output "zfs -V ". The module number does not match the utils number.

right

vperas · 13. März 2024

Zitat von Krisbee

Something still amiss in the output "zfs -V ". The module number does not match the utils number.

I appreciate your input, thanks!

Do you think I should revert back to OMV6 (from backup), or can I use this for now; everything seems to be functioning OK - filesystem, shares, docker...?

marc_al · 13. März 2024

Hello,

I am still stuck at the first part

When I remove the old config files my zfs pool is not mounted anymore (I can force import on the GUI) but U gave an error

Code

00 - Internal Server Error
Fatal error 45: Comment not terminated (line=1072, column=51); Fatal error 74: EndTag: '</' not found (line=1072, column=51)

I can't copy the other error (not fast enough). but talks about duplicate

And on reboot it is not imported anymore.

Do you have an idea? Where can I look for a way to automount (before docker).

Thank you

Krisbee · 13. März 2024

Zitat von vperas

I appreciate your input, thanks!

Do you think I should revert back to OMV6 (from backup), or can I use this for now; everything seems to be functioning OK - filesystem, shares, docker...?

No. Now I'm back I checked proxmox itself and I was surprised to see that "zfs -V" the output was in fact this:

Code

root@pveuefi:~# uname -a       
Linux pveuefi 6.5.13-1-pve #1 SMP PREEMPT_DYNAMIC PMX 6.5.13-1 (2024-02-05T13:50Z) x86_64 GNU/Linux

root@pveuefi:~# zfs -V
zfs-2.2.2-pve2
zfs-kmod-2.2.2-pve1
root@pveuefi:~#

So your system is OK. If you had upgraded from OMV6 to OMV7 starting with zfs and a pve kernel using omv-release-upgrade then you upgrade appears to have succeeded when others have failed.

PVE Kernel removed going from 6 to 7 and can't reinstall

Jetzt mitmachen!

Tags