Upgrade Scripts for non-interactive major release upgrades (2->3, 3->4, 4->5)

  • EDIT: ATTENTION-Before attempting an upgrade please check if the repo of omv-extras has been updated to github (previously it was on bintray) on your installation:

    omv-extras repos moving to github

    for OMV4 look at post #9


    I recently published some information on how to attempt an upgrade from OMV2 to OMV3 including the upgrade to Debian Jessie. I have now written scripts to automate this process for all three major upgrade paths: 2 to 3, 3 to 4, and 4 to 5.


    https://github.com/dleidert/openmediavault-upgrade


    I put all the knowledge and experience I collected in these scripts as well as tests and fixes for issues to expect as announced by the Debian release notes for each release. The scripts are based on the original omv-release-upgrade script supplied by openmediavault, but are much more sophisticated. I've successfully tested them in different constellations in VMs and managed the successful upgrade from 2 to 3 to 4 to 5. Note, that I cannot test all possible system variations though. So if you are interested in upgrading your system with these scripts and you have an unusual configuration you can let me know beforehand and I'll do my best to recreate the configuration in a VM and test it.


    Note that an upgrade attempt of an OMV2 system will only be possible as long as Debian provides repositories for Debian Jessie (8). After the repositories are gone the upgrade path will be closed and you will have to reinstall from scratch.


    Before you attempt any upgrade I recommend the following steps:


    1. Make a full backup of your operating system and make sure you know how to play it back in case something awful happens.
    2. Uninstall all unused plugins. Those make the upgrade harder. But if you don't use/need them then don't install them.
    3. Disable all services running on the OMV machine except for SSH. This will prevent any users from trying to access these services during the upgrade process and it will also allow to remount the data partitions/devices with the shared folders, samba shares, etc. as read-only before attempting the upgrade (mandatory at the moment)
    4. If the upgrade attempt is done remotely via SSH I highly recommend to install screen and run the script in a screen session (see below). Otherwise if the SSH connection breaks the console session will close too and the running script will be terminated even if it has not finished and leaves the system broken.
    5. If you are running encryption (especially on the boot device), if you have unusual boot configurations (scripts), if you are running graphical desktop environments, if you have a lot of third party packages installed, or if you want to stay with sysvinit instead of systemd please make sure you read the Debian release notes. In these cases you are probably better off with a manual upgrade or you have to prepare your system before you can run the scripts.


    The scripts are kept in branches for each openmediavault major series and are also released with the version accordingly to the major openmediavault version.


    OMV2 to OMV3

    Branch: https://github.com/dleidert/openmediavault-upgrade/tree/2

    Latest sources: https://github.com/dleidert/op…-upgrade/archive/2.tar.gz

    Current release: https://github.com/dleidert/op…grade/archive/v2.4.tar.gz

    Script name: `omv-release-upgrade-3`

    Debian release notes: https://www.debian.org/releases/jessie/releasenotes

    OMV3 to OMV4

    Branch: https://github.com/dleidert/openmediavault-upgrade/tree/3

    Latest sources: https://github.com/dleidert/op…-upgrade/archive/3.tar.gz

    Current release: https://github.com/dleidert/op…grade/archive/v3.3.tar.gz

    Script name: `omv-release-upgrade-4`

    Debian release notes: https://www.debian.org/releases/stretch/releasenotes

    OMV4 to OMV5

    Branch: https://github.com/dleidert/openmediavault-upgrade/tree/4

    Latest sources: https://github.com/dleidert/op…-upgrade/archive/4.tar.gz

    Current release: https://github.com/dleidert/op…grade/archive/v4.7.tar.gz

    Script name: `omv-release-upgrade-5`

    Debian release notes: https://www.debian.org/releases/buster/releasenotes

    Installing Screen

    Debian Wheezy users (OMV2) have to get the binary package for their architecture from snapshot.debian.org and then install it. Here for amd64:


    sudo dpkg -i screen_4.1.0~20120320gitdb59704-7+deb7u1_amd64.deb


    Users of later Debian versions can get it via


    sudo apt-get install screen


    The README.md file supplied with the tarball contains some basic information on how to use screen. But there is plenty of easy to understand documentation out there.

    Checking the correct boot device

    There is an issue with grub which can break your system after an unattended upgrade procedure. If devices have been changed on the target system the target device to install the grub boot-loader into stored in the debconf database might be wrong. Please test that the device given by


    debconf-show grub-pc | grep -w grub-pc/install_devices


    is actually the device the grub boot loader is intended to be installed to. If that's not the case run


    sudo dpkg-reconfigure -plow grub-pc


    and update the device. Otherwise you might end up not being able to boot your system just being shown the following error message:


    grub-pc: error: symbol `grub_calloc' not found


    If that is the case get a Debian installation CD/USB image, boot it, choose rescue mode, and follow the instructions to re-install the grub boot loader into the correct destination.

    Running the script

    One needs to get the tarball, unpack it, and run the script. The following shows it for the attempt to upgrade an OMV2 system:


    wget -qO- https://github.com/dleidert/openmediavault-upgrade/archive/v2.4.tar.gz | tar -xz

    cd openmediavault-upgrade-2.4

    sudo ./omv-release-upgrade-3


    If anything goes wrong there is a logfile created by the script which you can supply. The logfile is created in the directory from which you run the script and is called omv_release_upgrade_to_<debian_codename>.<date+time>.log. If everything goes to plan there will be a message telling you to restart the system.


    For the OMV 2 to 3 upgrade path it is vital that you check your /etc/fstab for devices with an auto mount option but no nofail option before you reboot. Such devices must be present during boot or the system won't boot successfully. There is a script to add the nofail attribute to these mountpoints during the upgrade. However, make double sure before rebooting.

    Feedback

    Feel free to send me feedback or report issues.

    16 Mal editiert, zuletzt von macom () aus folgendem Grund: Added note regarding OMV-extras repo as this is causing lots of issues

  • First impression is great, do you consider a PR to official repository?

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

    • Offizieller Beitrag

    That is a lot of work. Hopefully it helps people. I will change my post to point to your scripts.


    I recommend removing omv-extras before dist-upgrade and re-install after (at least with the 4 to 5 upgrade). Also, if you don't remove omv-extras plugins before the 2 to 3 upgrade, omv-engined will most likely not start causing the upgrade to fail.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    Einmal editiert, zuletzt von ryecoaaron ()

    • Offizieller Beitrag

    do you consider a PR to official repository?

    There is currently no omv-extras code in the official repository (not my choice). These scripts have omv-extras-related items in them.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I recommend removing omv-extras before dist-upgrade and re-install after.

    I have thought about that. It requires some magic because it shall only be attempted if omv-extras had been installed. If you think this is urgent I will handle it (to be honest the 4to5 script is the least tested one - too many possible variations). But the script actually runs through even in the current order (the same you used).


    Also, if you don't remove omv-extras plugins before the 2 to 3 upgrade, omv-engined will most likely not start causing the upgrade to fail.

    Has there been an omv-extras in OMV2? Shall this be removed in pre.d/50-remove-plugins?

    • Offizieller Beitrag

    I have thought about that. It requires some magic because it shall only be attempted if om-vextras had been installed. If you think this is urgent I will handle it (to be honest the 4to5 script is the least tested one - too many possible variations). But the script actually runs through even in the current order (the same you used).

    omv-extras 2.x and 3.x had pre upgrade scripts to try to do the right thing. Since OMV 4 never had the omv-release-upgrade framework, omv-extras 4.x does not have these files.

    Has there been an omv-extras in OMV2?

    omv-extras has been around since OMV 0.5.


    Shall this be removed in pre.d/50-remove-plugins

    I would have to go back and look. I haven't dealt with that code or the 2.x plugins in a long time.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • The pre.d/50-remove-plugins script in the OMV2 branch actually removes every package starting with openmediavault except for openmediavault and openmediavault-keyring. So if these plugins don't have any unusual names the script already cares about them.


    For the OMV 3 to 4 path I've read that removal of the plugins wasn't necessary. Therefor this script is disabled for this path. I'd need at least a list of plugins to remove. Or I could remove everything named openmediavault* but not coming from the official repository.

    • Offizieller Beitrag

    The pre.d/50-remove-plugins script in the OMV2 branch actually removes every package starting with openmediavault except for openmediavault and openmediavault-keyring. So if these plugins don't have any unusual names the script already cares about them.

    Most of these plugins were available for OMV 2.x - https://omv-extras.org/debian/pool/main/o/


    For the OMV 3 to 4 path I've read that removal of the plugins wasn't necessary. Therefor this script is disabled for this path. I'd need at least a list of plugins to remove. Or I could remove everything named openmediavault* but not coming from the official repository.

    There are some differences between OMV 3.x and 4.x plugins. I know personally, I usually didn't increment the plugin version from 3.x to 4.x if it worked on 3.x and 4.x. If the plugin is version 4.x, it most likely will not work on OMV 3.x.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hey dleidert, thanks for your awesome upgrade scripts. These really seem like a lot of work. I absolutely love that there are scripts automating this process.


    Now maybe you can help me with a little problem upgrading from 4 to 5. The upgrade script failed giving me this error message in the Log.



    I figured I google the issue finding out that people had this problem before upgrading manually and were able to fix this by commenting out the following line in the omg post install script /var/lib/dpkg/info/openmediavault.postinst

    Code
    omv-salt deploy run --no-color --quiet collectd chrony postfix


    After that running that command manually, apt-get update and apt-get upgrade works fine. This all worked out fine but im not getting a GUI (with cleared cache).


    Here the summary with the two failed below:


    Summary for test.local

    -------------

    Succeeded: 11 (changed=4)

    Failed: 2

    -------------

    Total states run: 13

    Total run time: 2.695 s




    Is there a possibility to just run the post install script again or should this be done another way?


    Maybe you have an idea what went wrong. Once I fully understand what is happening I can maybe create a pull request for you. Right now im still a little overwhelmed.

  • UPDATE: The GUI-login now suddenly showed up without any changes. After login with the following error:


    • Offizieller Beitrag

    Failed to open include file 'js/omv/PluginManager.js'.

    You still have an OMV 4.x plugin installed. What is the output of: dpkg -l | grep openm

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • You still have an OMV 4.x plugin installed. What is the output of: dpkg -l | grep openm

    Ohh yes you are very much correct indeed. I did remove all of them that I was able to. After:


    Code
    it  openmediavault                      5.5.20-1                              all          openmediavault - The open network attached storage solution
    rH  openmediavault-diskstats            5.0.6-1                               all          openmediavault disk monitoring plugin
    ii  openmediavault-keyring              1.0                                   all          GnuPG archive keys of the OpenMediaVault archive
    ic  openmediavault-omvextrasorg         4.1.16                                all          OMV-Extras.org Package Repositories for OpenMediaVault


    Unfortunately I can't remove openmediavault-omvextrasorg with apt-get --purge remove openmediavault-omvextrasorg because:

    Code
    Errors were encountered while processing: openmediavault-diskstats
    • Offizieller Beitrag

    Unfortunately I can't remove openmediavault-omvextrasorg with apt-get --purge remove openmediavault-omvextrasorg because:

    I would try the following:


    sudo rm /var/lib/dpkg/info/openmediavault-diskstats.postrm

    sudo apt-get remove openmediavault-diskstats

    sudo apt-get purge openmediavault-omvextrasorg

    sudo omv-salt deploy run collectd

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • That worked perfectly!


    Only one failed in the summary:




    • Offizieller Beitrag

    You may need to make some tweaks to your notifications and network interface setup from the web interface to fix that.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!


  • I figured I google the issue finding out that people had this problem before upgrading manually and were able to fix this by commenting out the following line in the omg post install script /var/lib/dpkg/info/openmediavault.postinst

    Code
    omv-salt deploy run --no-color --quiet collectd chrony postfix

    Can someone explain the cause of this issue to me? I searched through the forum but I don't quite get it. It doesn't seem to make a difference if I enable or disable NTP.


    Ok. After some more reading my impression is that chronyd starts to fail on systems running a kernel with SECCOMP disabled, because the default /etc/default/chrony runs it with seccomp enabled via the -F 1 switch. So I can workaround this by supplying a a configuration file beforehand which disabled the seccomp filter. But I wonder if openmediavault's postinst (and omv-salt) should better take care of this.

    • Offizieller Beitrag

    I have a workaround to deal with it.

    This won't work on RPi systems since they don't put their config file in /boot. You can modprobe configs and zgrep the config from /proc/config.gz though. But I am curious what kernel doesn't have SECCOMP enabled? Debian, proxmox, rpi, and armbian kernels have it enabled.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Custom kernels I guess. I asked the poster who had the issue for kernel information, but didn't receive an answer so far. With the new releases the log file will contain information about the kernel.


    Debian systems don't make the config available in /proc/config*. That's why I went with /boot. But I can make a really conservative approach:


    1. test for /proc/config* and use it, else
    2. test for /boot/config-`uname -r` and use it, else
    3. assume seccomp is not avaialble and go with -F 0 (which is the default anyway)

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!