Upgrade Scripts for non-interactive major release upgrades (2->3, 3->4, 4->5)

  • mark a defined list of packages as manually installed

    A good starting point, but I'd fear it will never be able to match with all the customizations done by OMV NAS builders.

    Good ideas how to overcome this limitation are needed

    omv 5.6.3-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • dleidert I'm seeing the same initial issue as Alen saw:

    Code
    Unpacking usrmerge (21) ...
    Setting up libtext-glob-perl (0.10-1) ...
    Setting up libnumber-compare-perl (0.03-1) ...
    Setting up libfile-find-rule-perl (0.34-1) ...
    Setting up usrmerge (21) ...
    The system has been successfully converted.
    Removed 'openmediavault-nut ' because of file conflicts affecting the 'usrmerge' transition.
    run-parts: executing /root/openmediavault-upgrade-4.4/post.d/70-omv-salt
    /root/openmediavault-upgrade-4.4/post.d/70-omv-salt: 12: /root/openmediavault-upgrade-4.4/post.d/70-omv-salt: omv-salt: not found
    run-parts: /root/openmediavault-upgrade-4.4/post.d/70-omv-salt exited with return code 127


    I'm using openmediavault-upgrade-4.4 and I tried following your suggestions from #50, but I'm not making any progress:


    Hoping to get some guidance on how to proceed. Thank you in advance for all the time you've invested in these scripts and helping the community.

  • A good starting point, but I'd fear it will never be able to match with all the customizations done by OMV NAS builders.

    Good ideas how to overcome this limitation are needed

    Well, IMO you misjudge some things here.


    Debian itself provides an endless combination of packages and configurations. It is far more diverse then OMV installations are. Still Debian provides a tested and documented way to upgrade every system using apt and the Debian release notes. All packages provided by the Debian project are actually obligated to provide an upgrade path from one stable release to the next. (Almost) All possible obstacles are listed in the release notes. Upgrades are tested well before a new Debian stable release goes live. So all the tools used, e.g. docker-ce, are upgradeable. Now if openmediavault would be an official part of the packages provided by the project it would have to fulfill this requirement too.


    Now here comes the problem: It isn't and it doesn't. The same goes for other packages provided by the omv ecosystem. So why do we have to remove openmediavault-docker-gui manually when openmediavault (>= 5) just integrated the plugin's functionality: because it doesn't declare the necessary Provides, Conflicts, or Replaces control fields. Why are plugins not removed automatically when they are incompatible with a new major omv release? Because they don't declare that they depend on openmediavault (<< next_major_release) or declare an API compatibility like a lot of tools in the Debian archive. The issue fixed with release 4.4 actually is caused by omv using its codename as distribution name for the repository and not the Debian codenames ("usul" instead of "buster"). And the list goes on. So that's what these scripts do: they try to work around the limitations the omv package ecosystem comes with and they script what every Debian release note in section 4 suggests.


    But for this purpose to succeed the possible customizations of OMV systems are not really important. Debian packages all work the same way, the basic limitations are the same for all OMV users, and a Debian upgrade also always consists of the same steps. So that's why (in my opinion) we can generalize most issues and OMV systems.


    Of course there are limitations and IMHO I am very clear about them. But then again: If you have a highly customized system with unusual boot scripts, kernels, or third party sources mixed in (like Ubuntu PPAs) you are always on your own. If you do this you should read the Debian release notes, you should be able to manually upgrade your system (still the scripts can guide you), and you should be able to handle all problems occurring.

  • I'm using openmediavault-upgrade-4.4 and I tried following your suggestions from #50, but I'm not making any progress:

    [..]

    Hoping to get some guidance on how to proceed. Thank you in advance for all the time you've invested in these scripts and helping the community.

    I come back to you in 15 minutes. I'm running the situation in a VM. One probably has to run some more scripts from the pre.d directory. Let me check quickly

  • evanfromiowa I've corrected the instructions in #50. It basically boils down to this (use at least the 4.4 release):

    Code
    . inc/envvars
    sudo pre.d/01-create-omvvars
    sudo pre.d/72-apt-update-omv-source-list
    sudo pre.d/73-apt-source-remove-docker
    sudo pre.d/75-apt-source-armbian-fix
    sudo run.d/10-apt-update
    sudo run.d/80-apt-dist-upgrade
    sudo run.d/81-apt-dist-upgrade-omv
    sudo run-parts -v --exit-on-error post.d

    This will install the firmware and kernel packages from buster-backports if you have the backports enabled. To avoid it temporarily comment out the contents in /etc/apt/sources.list.d/openmediavault-kernel-backports.list before you do the above steps. However this should fix your system.

  • evanfromiowa I've corrected the instructions in #50. It basically boils down to this (use at least the 4.4 release):

    Code
    . inc/envvars
    sudo pre.d/01-create-omvvars
    sudo pre.d/72-apt-update-omv-source-list
    sudo pre.d/73-apt-source-remove-docker
    sudo pre.d/75-apt-source-armbian-fix
    sudo run.d/10-apt-update
    sudo run.d/80-apt-dist-upgrade
    sudo run.d/81-apt-dist-upgrade-omv
    sudo run-parts -v --exit-on-error post.d

    This will install the firmware and kernel packages from buster-backports if you have the backports enabled. To avoid it temporarily comment out the contents in /etc/apt/sources.list.d/openmediavault-kernel-backports.list before you do the above steps. However this should fix your system.

    Sorry, just want to clarify before attempting. Are you say I do need to comment out /etc/apt/sources.list.d/openmediavault-kernel-backports.list or that I do not need to comment out the backports.list file before running the steps you've mentioned?

  • Sorry, just want to clarify before attempting. Are you say I do need to comment out /etc/apt/sources.list.d/openmediavault-kernel-backports.list or that I do not need to comment out the backports.list file before running the steps you've mentioned?

    Comment out the contents of this file (set a # in front of the entry in the file) if you do not want to install the firmware and kernel packages from buster-backports during the upgrade. If you are ok with that or if you haven't enabled the backports you don't need to touch the file.

  • vcdwelt, I tested the scripts in the upcoming 4.5 release with a VM running all the packages you gave me. The upgrade was successful. However plexmediaserver gets removed at the end because the plex repository is removed. From my understanding in OMV5 you are supposed to run Plex via docker/portainer. The plugin doesn't exist anymore and without manual interaction you won't have any repository for plexmediaserver either. So I'm not even sure if it is even a good idea to keep the plexmediaserver package on the system. ryecoaaron what do you think?

  • evanfromiowa I've corrected the instructions in #50. It basically boils down to this (use at least the 4.4 release):

    Code
    . inc/envvars
    sudo pre.d/01-create-omvvars
    sudo pre.d/72-apt-update-omv-source-list
    sudo pre.d/73-apt-source-remove-docker
    sudo pre.d/75-apt-source-armbian-fix
    sudo run.d/10-apt-update
    sudo run.d/80-apt-dist-upgrade
    sudo run.d/81-apt-dist-upgrade-omv
    sudo run-parts -v --exit-on-error post.d

    This will install the firmware and kernel packages from buster-backports if you have the backports enabled. To avoid it temporarily comment out the contents in /etc/apt/sources.list.d/openmediavault-kernel-backports.list before you do the above steps. However this should fix your system.

    Ran all of this (without commenting out backports) and I’m now running OMV 5.5.21-1 on Debian buster! Thanks dleidert for the help on this fix and all your great work putting the whole openmediavault-upgrade repo together.

  • So why do we have to remove openmediavault-docker-gui manually when openmediavault (>= 5) just integrated the plugin's functionality: because it doesn't declare the necessary Provides, Conflicts, or Replaces control fields. Why are plugins not removed automatically when they are incompatible with a new major omv release? Because they don't declare that they depend on openmediavault (<< next_major_release) or declare an API compatibility like a lot of tools in the Debian archive. The issue fixed with release 4.4 actually is caused by omv using its codename as distribution name for the repository and not the Debian codenames ("usul" instead of "buster"). And the list goes on. So that's what these scripts do: they try to work around the limitations the omv package ecosystem comes with and they script what every Debian release note in section 4 suggests.

    Thanks for clarification

    omv 5.6.3-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • vcdwelt, I tested the scripts in the upcoming 4.5 release with a VM running all the packages you gave me. The upgrade was successful. However plexmediaserver gets removed at the end because the plex repository is removed. From my understanding in OMV5 you are supposed to run Plex via docker/portainer. The plugin doesn't exist anymore and without manual interaction you won't have any repository for plexmediaserver either. So I'm not even sure if it is even a good idea to keep the plexmediaserver package on the system. ryecoaaron what do you think?

    For me it is not a big thing if the Plex Part is removed out of the OMV. I just want to keep my Plex installed as it is. I can maintain it without the Plugin Part from OMV. Maybe there is anything i could do to prevent the uninstallation of Plex.
    Or i'm completely on the wrong way thinking about this problem?

  • So why do we have to remove openmediavault-docker-gui manually when openmediavault (>= 5) just integrated the plugin's functionality: because it doesn't declare the necessary Provides, Conflicts, or Replaces control fields.

    openmediavault-docker-gui is a plugin which happens to install docker-ce.. omv-extras 5.x just installs docker-ce and can create containers for portainer and yacht. Big difference. omv-extras does not replace or conflict with the plugin and shouldn't be marked as providing docker.

    Why are plugins not removed automatically when they are incompatible with a new major omv release? Because they don't declare that they depend on openmediavault (<< next_major_release)

    I have played that game before and too many times, it has broken upgrades (not just release upgrades). While I agree plugins are debian packages, they have a much tighter link to openmediavault than most packages.

    omv 5.6.4 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • For me it is not a big thing if the Plex Part is removed out of the OMV. I just want to keep my Plex installed as it is. I can maintain it without the Plugin Part from OMV. Maybe there is anything i could do to prevent the uninstallation of Plex.
    Or i'm completely on the wrong way thinking about this problem?

    Well, you can prevent it from being removed. Get the scripts. Before you start the main script, open post.d/85-apt-autoremove-cruft in an editor of your choice. Look for the line starting with


    apt-show-versions | grep 'No available version' [..]


    and change it so it reads


    apt-show-versions | grep 'No available version' | awk -F: '{print $1}' | grep -x -v -e ${currkern} -e plexmediaserver | xargs apt-get autoremove --purge --yes


    This will prevent plexmediaserver from being removed at this point.


    Edit: For versions up to 4.5 the line


    currkern="linux-image-`uname -r`+"


    must be changed to


    currkern="linux-image-`uname -r`"


    as well.


    Edit: Actually this is not necessary. Enabling the plex repository before running the script will prevent plexmediaserver from being listed by apt-show-versions and thus prevent its removal.




  • Afterwards you might want to add a repository for plexmediaserver suitable for buster. Possibly create /etc/apt/sources.list.d/plexmediaserver.list and put in a the resource. Then run apt-get update and apt-get dist-upgrade.


    I have to say though I have no idea if the plexmediaserver version you are running will start with the Buster operating system. So if I were in your shoes I would probably do this before the reboot.


    Edit: Actually this is not necessary. Enabling the plex repository before running the script will prevent plexmediaserver from being listed by apt-show-versions and thus prevent its removal. It will also pull in any update available.

  • Thank you again for the feedback. I have already added the plex repo and updating the Plex Server trough apt. It also should be running with buster as far i read in Plex Forum already.

    I have a Backup done, if anything will go wrong, hopefully the Backup does his Job :-D

  • vcdwelt , there is another line to change:


    currkern="linux-image-`uname -r`+"


    must become


    currkern="linux-image-`uname -r`"


    in the same file. Sorry.


    Edit: Actually this is not necessary. Enabling the plex repository before running the script will prevent plexmediaserver from being listed by apt-show-versions and thus prevent its removal.

  • vcdwelt , I just released version 4.6. Either enable the plex repository in /etc/apt/sources.list.d/plexmediaserver.list` before running the upgrade script or edit inc/envvars and uncomment the export THIS_UPGRADE_KEEP_PLEX=1 (will basically do the same). This keeps the plexmediaserver package on the system.

  • dleidert / all,


    I've now upgraded my live OMV 4 system using your 4.6 scripts and ran into following issue.

    I can login to OMV where is see the message "The configuration has been changed. You must apply the changes in order for them to take effect." in the yellow banner.

    Whenever I click on APPLY I get following error message :-( Does somebody understand what this error message is about?



    Code
    {"response":null,"error":{"code":0,"message":"","trace":"OMV\\ExecException: Failed to execute command 'export PATH=\/bin:\/sbin:\/usr\/bin:\/usr\/sbin:\/usr\/local\/bin:\/usr\/local\/sbin; export LANG=C.UTF-8; omv-salt deploy run --no-color rsync 2>&1' with exit code '1': omv.local:\n----------\n ID: prereq_rsync_certificates\n Function: salt.state\n Result: True\n


    The error message is too long to be posted - so find it as text file here :ErrorMessageApply.txt



    I've also attached the log file from your update scripts.

    omv_release_upgrade_to_buster.2021-01-08-235830.log.txt


    EDIT: I found the solution myself. I had to remove a dodgy/invalid Remote Share. This left me with another issue - which was gone when I cleared the browser cache. So it's really a good idea to clear browser cache after the upgrade !!

  • Thank you for sharing!


    However, I tried to upgrade my NanoPi M4 from 4 -> 5 and even though the upgrade seemed to be successful, when I rebooted my device (as told after the script finished), now it has no network (I don't even see it listed in my router's list of clients) and I have no output via HDMI. Guess it's dead and I will have to do a clean install 🤷🏼

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!