Upgrade Scripts for non-interactive major release upgrades (2->3, 3->4, 4->5)

  • I recently published some information on how to attempt an upgrade from OMV2 to OMV3 including the upgrade to Debian Jessie. I have now written scripts to automate this process for all three major upgrade paths: 2 to 3, 3 to 4, and 4 to 5.


    https://github.com/dleidert/openmediavault-upgrade


    I put all the knowledge and experience I collected in these scripts as well as tests and fixes for issues to expect as announced by the Debian release notes for each release. The scripts are based on the original omv-release-upgrade script supplied by openmediavault, but are much more sophisticated. I've successfully tested them in different constellations in VMs and managed the successful upgrade from 2 to 3 to 4 to 5. Note, that I cannot test all possible system variations though. So if you are interested in upgrading your system with these scripts and you have an unusual configuration you can let me know beforehand and I'll do my best to recreate the configuration in a VM and test it.


    Note that an upgrade attempt of an OMV2 system will only be possible as long as Debian provides repositories for Debian Jessie (8). After the repositories are gone the upgrade path will be closed and you will have to reinstall from scratch.


    Before you attempt any upgrade I recommend the following steps:


    1. Make a full backup of your operating system and make sure you know how to play it back in case something awful happens.
    2. Uninstall all unused plugins. Those make the upgrade harder. But if you don't use/need them then don't install them.
    3. Disable all services running on the OMV machine except for SSH. This will prevent any users from trying to access these services during the upgrade process and it will also allow to remount the data partitions/devices with the shared folders, samba shares, etc. as read-only before attempting the upgrade (mandatory at the moment)
    4. If the upgrade attempt is done remotely via SSH I highly recommend to install screen and run the script in a screen session (see below). Otherwise if the SSH connection breaks the console session will close too and the running script will be terminated even if it has not finished and leaves the system broken.
    5. If you are running encryption (especially on the boot device), if you have unusual boot configurations (scripts), if you are running graphical desktop environments, if you have a lot of third party packages installed, or if you want to stay with sysvinit instead of systemd please make sure you read the Debian release notes. In these cases you are probably better off with a manual upgrade or you have to prepare your system before you can run the scripts.


    The scripts are kept in branches for each openmediavault major series and are also released with the version accordingly to the major openmediavault version.


    OMV2 to OMV3

    Branch: https://github.com/dleidert/openmediavault-upgrade/tree/2

    Latest sources: https://github.com/dleidert/op…-upgrade/archive/2.tar.gz

    Current release: https://github.com/dleidert/op…grade/archive/v2.4.tar.gz

    Script name: `omv-release-upgrade-3`

    Debian release notes: https://www.debian.org/releases/jessie/releasenotes

    OMV3 to OMV4

    Branch: https://github.com/dleidert/openmediavault-upgrade/tree/3

    Latest sources: https://github.com/dleidert/op…-upgrade/archive/3.tar.gz

    Current release: https://github.com/dleidert/op…grade/archive/v3.3.tar.gz

    Script name: `omv-release-upgrade-4`

    Debian release notes: https://www.debian.org/releases/stretch/releasenotes

    OMV4 to OMV5

    Branch: https://github.com/dleidert/openmediavault-upgrade/tree/4

    Latest sources: https://github.com/dleidert/op…-upgrade/archive/4.tar.gz

    Current release: https://github.com/dleidert/op…grade/archive/v4.7.tar.gz

    Script name: `omv-release-upgrade-5`

    Debian release notes: https://www.debian.org/releases/buster/releasenotes

    Installing Screen

    Debian Wheezy users (OMV2) have to get the binary package for their architecture from snapshot.debian.org and then install it. Here for amd64:


    sudo dpkg -i screen_4.1.0~20120320gitdb59704-7+deb7u1_amd64.deb


    Users of later Debian versions can get it via


    sudo apt-get install screen


    The README.md file supplied with the tarball contains some basic information on how to use screen. But there is plenty of easy to understand documentation out there.

    Running the script

    One needs to get the tarball, unpack it, and run the script. The following shows it for the attempt to upgrade an OMV2 system:


    wget -qO- https://github.com/dleidert/openmediavault-upgrade/archive/v2.3.tar.gz | tar -xz

    cd openmediavault-upgrade-2.3

    sudo ./omv-release-upgrade-3


    If anything goes wrong there is a logfile created by the script which you can supply. The logfile is created in the directory from which you run the script and is called omv_release_upgrade_to_<debian_codename>.<date+time>.log. If everything goes to plan there will be a message telling you to restart the system.


    For the OMV 2 to 3 upgrade path it is vital that you check your /etc/fstab for devices with an auto mount option but no nofail option before you reboot. Such devices must be present during boot or the system won't boot successfully. There is a script to add the nofail attribute to these mountpoints during the upgrade. However, make double sure before rebooting.

    Feedback

    Feel free to send me feedback or report issues.

  • That is a lot of work. Hopefully it helps people. I will change my post to point to your scripts.


    I recommend removing omv-extras before dist-upgrade and re-install after (at least with the 4 to 5 upgrade). Also, if you don't remove omv-extras plugins before the 2 to 3 upgrade, omv-engined will most likely not start causing the upgrade to fail.

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

    Edited once, last by ryecoaaron ().

  • do you consider a PR to official repository?

    There is currently no omv-extras code in the official repository (not my choice). These scripts have omv-extras-related items in them.

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I recommend removing omv-extras before dist-upgrade and re-install after.

    I have thought about that. It requires some magic because it shall only be attempted if omv-extras had been installed. If you think this is urgent I will handle it (to be honest the 4to5 script is the least tested one - too many possible variations). But the script actually runs through even in the current order (the same you used).


    Also, if you don't remove omv-extras plugins before the 2 to 3 upgrade, omv-engined will most likely not start causing the upgrade to fail.

    Has there been an omv-extras in OMV2? Shall this be removed in pre.d/50-remove-plugins?

  • I have thought about that. It requires some magic because it shall only be attempted if om-vextras had been installed. If you think this is urgent I will handle it (to be honest the 4to5 script is the least tested one - too many possible variations). But the script actually runs through even in the current order (the same you used).

    omv-extras 2.x and 3.x had pre upgrade scripts to try to do the right thing. Since OMV 4 never had the omv-release-upgrade framework, omv-extras 4.x does not have these files.

    Has there been an omv-extras in OMV2?

    omv-extras has been around since OMV 0.5.


    Shall this be removed in pre.d/50-remove-plugins

    I would have to go back and look. I haven't dealt with that code or the 2.x plugins in a long time.

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • The pre.d/50-remove-plugins script in the OMV2 branch actually removes every package starting with openmediavault except for openmediavault and openmediavault-keyring. So if these plugins don't have any unusual names the script already cares about them.


    For the OMV 3 to 4 path I've read that removal of the plugins wasn't necessary. Therefor this script is disabled for this path. I'd need at least a list of plugins to remove. Or I could remove everything named openmediavault* but not coming from the official repository.

  • The pre.d/50-remove-plugins script in the OMV2 branch actually removes every package starting with openmediavault except for openmediavault and openmediavault-keyring. So if these plugins don't have any unusual names the script already cares about them.

    Most of these plugins were available for OMV 2.x - https://omv-extras.org/debian/pool/main/o/


    For the OMV 3 to 4 path I've read that removal of the plugins wasn't necessary. Therefor this script is disabled for this path. I'd need at least a list of plugins to remove. Or I could remove everything named openmediavault* but not coming from the official repository.

    There are some differences between OMV 3.x and 4.x plugins. I know personally, I usually didn't increment the plugin version from 3.x to 4.x if it worked on 3.x and 4.x. If the plugin is version 4.x, it most likely will not work on OMV 3.x.

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Hey dleidert, thanks for your awesome upgrade scripts. These really seem like a lot of work. I absolutely love that there are scripts automating this process.


    Now maybe you can help me with a little problem upgrading from 4 to 5. The upgrade script failed giving me this error message in the Log.



    I figured I google the issue finding out that people had this problem before upgrading manually and were able to fix this by commenting out the following line in the omg post install script /var/lib/dpkg/info/openmediavault.postinst

    Code
    omv-salt deploy run --no-color --quiet collectd chrony postfix


    After that running that command manually, apt-get update and apt-get upgrade works fine. This all worked out fine but im not getting a GUI (with cleared cache).


    Here the summary with the two failed below:


    Summary for test.local

    -------------

    Succeeded: 11 (changed=4)

    Failed: 2

    -------------

    Total states run: 13

    Total run time: 2.695 s




    Is there a possibility to just run the post install script again or should this be done another way?


    Maybe you have an idea what went wrong. Once I fully understand what is happening I can maybe create a pull request for you. Right now im still a little overwhelmed.

  • UPDATE: The GUI-login now suddenly showed up without any changes. After login with the following error:


  • You still have an OMV 4.x plugin installed. What is the output of: dpkg -l | grep openm

    Ohh yes you are very much correct indeed. I did remove all of them that I was able to. After:


    Code
    it openmediavault 5.5.20-1 all openmediavault - The open network attached storage solution
    rH openmediavault-diskstats 5.0.6-1 all openmediavault disk monitoring plugin
    ii openmediavault-keyring 1.0 all GnuPG archive keys of the OpenMediaVault archive
    ic openmediavault-omvextrasorg 4.1.16 all OMV-Extras.org Package Repositories for OpenMediaVault


    Unfortunately I can't remove openmediavault-omvextrasorg with apt-get --purge remove openmediavault-omvextrasorg because:

    Code
    Errors were encountered while processing: openmediavault-diskstats
  • Unfortunately I can't remove openmediavault-omvextrasorg with apt-get --purge remove openmediavault-omvextrasorg because:

    I would try the following:


    sudo rm /var/lib/dpkg/info/openmediavault-diskstats.postrm

    sudo apt-get remove openmediavault-diskstats

    sudo apt-get purge openmediavault-omvextrasorg

    sudo omv-salt deploy run collectd

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • That worked perfectly!


    Only one failed in the summary:




  • You may need to make some tweaks to your notifications and network interface setup from the web interface to fix that.

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!


  • I figured I google the issue finding out that people had this problem before upgrading manually and were able to fix this by commenting out the following line in the omg post install script /var/lib/dpkg/info/openmediavault.postinst

    Code
    omv-salt deploy run --no-color --quiet collectd chrony postfix

    Can someone explain the cause of this issue to me? I searched through the forum but I don't quite get it. It doesn't seem to make a difference if I enable or disable NTP.


    Ok. After some more reading my impression is that chronyd starts to fail on systems running a kernel with SECCOMP disabled, because the default /etc/default/chrony runs it with seccomp enabled via the -F 1 switch. So I can workaround this by supplying a a configuration file beforehand which disabled the seccomp filter. But I wonder if openmediavault's postinst (and omv-salt) should better take care of this.

  • I have a workaround to deal with it.

    This won't work on RPi systems since they don't put their config file in /boot. You can modprobe configs and zgrep the config from /proc/config.gz though. But I am curious what kernel doesn't have SECCOMP enabled? Debian, proxmox, rpi, and armbian kernels have it enabled.

    omv 5.5.23 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Custom kernels I guess. I asked the poster who had the issue for kernel information, but didn't receive an answer so far. With the new releases the log file will contain information about the kernel.


    Debian systems don't make the config available in /proc/config*. That's why I went with /boot. But I can make a really conservative approach:


    1. test for /proc/config* and use it, else
    2. test for /boot/config-`uname -r` and use it, else
    3. assume seccomp is not avaialble and go with -F 0 (which is the default anyway)

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!