Upgrade Scripts for non-interactive major release upgrades (2->3, 3->4, 4->5)

    • Official Post

    test for /proc/config* and use it, else

    The problem with that is that the module is not loaded by default. So, the config will never exist unless something loads the module.


    go with -F 0 (which is the default anyway)

    DAEMON_OPTS="-F -1" is the default in buster.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.7 | compose 7.0.8 | cputemp 7.0 | mergerfs 7.0.2


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • The problem with that is that the module is not loaded by default. So, the config will never exist unless something loads the module.

    I think this now handles /proc/config* as well.


    DAEMON_OPTS="-F -1" is the default in buster.

    That's the content of the /etc/default/chrony file shipped with the package. What I meant is that if one doesn't set the -F option at all -F 0 would be the default behavior according to the manual page. So the chronyd maintainer made a decision to not go with the default behavior but to require seccomp support by default. And if -F 0 is upstream's default I don't see much harm in using it even if the script might apply the fix to some false-positives. The README.md file contains a hint about the change and how to revert it.

    • Official Post

    I think this now handles /proc/config* as well.

    agreed.


    I don't see much harm in using it even if the script might apply the fix to some false-positives

    Sounds ok to me.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.7 | compose 7.0.8 | cputemp 7.0 | mergerfs 7.0.2


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • hi dleidert


    many thanks for all your hard work putting these scripts together.


    I was trying to upgrade from 4.1.36-1 to 5.x by running


    omv-release-upgrade-5


    I'm getting this error at the end:


    Code
    Setting up usrmerge (21) ...
    The system has been successfully converted.
    Removed 'openmediavault-nut ' because of file conflicts affecting the 'usrmerge' transition.
    run-parts: executing /root/openmediavault-upgrade-4.3/post.d/70-omv-salt
    /root/openmediavault-upgrade-4.3/post.d/70-omv-salt: 12: /root/openmediavault-upgrade-4.3/post.d/70-omv-salt: omv-salt: not found
    run-parts: /root/openmediavault-upgrade-4.3/post.d/70-omv-salt exited with return code 127

    Seems like omv-salt is missing. Shouldn't it be installed under /usr/sbin ? Can't find it there.


    Any help is much appreciated. I'm testing the upgrade on a VM - so I can fall back and test the upgrade again.


    Thanks

    Alen

    Edited once, last by Alen: adding extra information. ().

  • Ok. Same issue as seen before in a similar case. When it comes to the dist-upgrade step, suddenly apt doesn't show openmediavault as upgradeable anymore. So it doesn't get upgraded to version 5.


    I'd love to find the reason this time: Can you please give me the output of sudo apt-cache policy openmediavault at this point? What is shown when you do sudo apt-get dist-upgrade -u -o Debug::pkgProblemResolver=true --assume-no openmediavault+ (the --assume-no switch will prevent the upgrade from actually running).


    Edit: When I examined the package list I got the impression you used some pinning for stretch-backports. Otherwise you couldn't have some of the package versions installed without them getting updated. So you would help me a lot by providing /etc/apt/preferences and /etc/apt/preferences .d/ and the output of sudo apt-cache policy as well. The more I think about it the more I suspect you have a package pinned in a a way that prevents its removal stopping openmediavault from being upgraded.

  • Hi - tried to provide all the requested information.


    Quote

    Can you please give me the output of sudo apt-cache policy openmediavault at this point?


    Code
    sudo apt-cache policy openmediavault
    openmediavault:
    Installed: 4.1.36-1
    Candidate: 4.1.36-1
    Version table:
    *** 4.1.36-1 995
    995 http://packages.openmediavault.org/public arrakis/main amd64 Packages
    995 https://openmediavault.github.io/packages arrakis/main amd64 Packages
    100 /var/lib/dpkg/status


    Quote

    What is shown when you do sudo apt-get dist-upgrade -u -o Debug::pkgProblemResolver=true --assume-no openmediavault+

    sudo apt-get dist-upgrade -u -o Debug::pkgProblemResolver=true --assume-no openmediavault+Reading package lists... DoneBuilding dependency treeReading state information... Doneopenmediavault is already the newest version (4.1.36-1).Starting pkgProblemResolver with broken count: 0Starting 2 pkgProblemResolver with broken count: 0DoneCalculating upgrade... DoneThe following packages were automatically installed and are no longer required:dh-python e2fslibs libavcodec57 libavformat57 libavutil55 libbind9-140 libbluray1 libcomerr2 libdbus-glib-1-2 libdns162 libevent-2.0-5 libfile-copy-recursive-perl libicu57 libisc160 libisccc140 libisccfg140 libltdl7 liblvm2app2.2liblvm2cmd2.02 liblwres141 libntfs-3g871 libonig4 libperl5.24 libpython3.5-minimal libpython3.5-stdlib libswresample2 liburcu4 libva-drm1 libva-x11-1 libva1 libvpx4 libwebpmux2 libx264-148 libx265-95 python3-distutils python3-lib2to3python3.5 python3.5-minimal update-inetdUse 'sudo apt autoremove' to remove them.The following packages will be upgraded:minidlna1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.Need to get 0 B/152 kB of archives.After this operation, 0 B of additional disk space will be used.Do you want to continue? [Y/n] NAbort.

    Quote
    So you would help me a lot by providing /etc/apt/preferences

    Can't find this file/directory

    Quote
    /etc/apt/preferences .d/

    /etc/apt/preferences.d# ls -latotal 20drwxr-xr-x 2 root root 4096 Jan 5 21:11 .drwxr-xr-x 12 root root 4096 Jan 5 21:09 ..-rw-r--r-- 1 root root 1337 Jan 5 21:11 omv-extras-org-rw-r--r-- 1 root root 291 Jan 5 21:11 openmediavault-kernel-backports.pref-rw-r--r-- 1 root root 44 May 5 2017

    Quote

    and the output of [tt]sudo apt-cache policy


    sudo apt-cache policy
    Package files:
    100 /var/lib/dpkg/status
    release a=now
    500 http://debian.ethz.ch/debian stretch-updates/non-free amd64 Packages
    release o=Debian,a=oldstable-updates,n=stretch-updates,l=Debian,c=non-free,b=amd64
    origin debian.ethz.ch
    500 http://debian.ethz.ch/debian stretch-updates/main amd64 Packages
    release o=Debian,a=oldstable-updates,n=stretch-updates,l=Debian,c=main,b=amd64
    origin debian.ethz.ch
    500 http://security.debian.org/debian-security stretch/updates/non-free amd64 Packages
    release v=9,o=Debian,a=oldstable,n=stretch,l=Debian-Security,c=non-free,b=amd64
    origin security.debian.org
    500 http://security.debian.org/debian-security stretch/updates/contrib amd64 Packages
    release v=9,o=Debian,a=oldstable,n=stretch,l=Debian-Security,c=contrib,b=amd64
    origin security.debian.org
    500 http://security.debian.org/debian-security stretch/updates/main amd64 Packages
    release v=9,o=Debian,a=oldstable,n=stretch,l=Debian-Security,c=main,b=amd64
    origin security.debian.org
    500 http://debian.ethz.ch/debian stretch/main amd64 Packages
    release v=9.13,o=Debian,a=oldstable,n=stretch,l=Debian,c=main,b=amd64
    origin debian.ethz.ch
    995 https://openmediavault.github.io/packages arrakis/main amd64 Packages
    release o=openmediavault.org archive,a=arrakis,n=arrakis,l=openmediavault.org archive,c=main,b=amd64
    origin openmediavault.github.io
    995 http://packages.openmediavault.org/public arrakis/main amd64 Packages
    release o=openmediavault.org archive,a=arrakis,n=arrakis,l=openmediavault.org archive,c=main,b=amd64
    origin packages.openmediavault.org
    995 file:/var/cache/openmediavault/archives Packages
    release c=
    100 http://httpredir.debian.org/debian buster-backports/non-free amd64 Packages
    release o=Debian Backports,a=buster-backports,n=buster-backports,l=Debian Backports,c=non-free,b=amd64
    origin httpredir.debian.org
    100 http://httpredir.debian.org/debian buster-backports/contrib amd64 Packages
    release o=Debian Backports,a=buster-backports,n=buster-backports,l=Debian Backports,c=contrib,b=amd64
    origin httpredir.debian.org
    100 http://httpredir.debian.org/debian buster-backports/main amd64 Packages
    release o=Debian Backports,a=buster-backports,n=buster-backports,l=Debian Backports,c=main,b=amd64
    origin httpredir.debian.org
    500 https://dl.bintray.com/openmed…plugin-developers/arrakis stretch/main amd64 Packages
    release o=Bintray,a=stretch,n=stretch,l=Bintray,c=main,b=amd64
    origin dl.bintray.com
    500 https://deb.debian.org/debian buster-updates/non-free amd64 Packages
    release o=Debian,a=stable-updates,n=buster-updates,l=Debian,c=non-free,b=amd64
    origin deb.debian.org
    500 https://deb.debian.org/debian buster-updates/main amd64 Packages
    release o=Debian,a=stable-updates,n=buster-updates,l=Debian,c=main,b=amd64
    origin deb.debian.org
    990 https://deb.debian.org/debian-security buster/updates/non-free amd64 Packages
    release v=10,o=Debian,a=stable,n=buster,l=Debian-Security,c=non-free,b=amd64
    origin deb.debian.org
    990 https://deb.debian.org/debian-security buster/updates/main amd64 Packages
    release v=10,o=Debian,a=stable,n=buster,l=Debian-Security,c=main,b=amd64
    origin deb.debian.org
    990 https://deb.debian.org/debian buster/non-free amd64 Packages
    release v=10.7,o=Debian,a=stable,n=buster,l=Debian,c=non-free,b=amd64
    origin deb.debian.org
    990 https://deb.debian.org/debian buster/contrib amd64 Packages
    release v=10.7,o=Debian,a=stable,n=buster,l=Debian,c=contrib,b=amd64
    origin deb.debian.org
    990 https://deb.debian.org/debian buster/main amd64 Packages
    release v=10.7,o=Debian,a=stable,n=buster,l=Debian,c=main,b=amd64
    origin deb.debian.org
    Pinned packages:
    linux-image-5.8.0-0.bpo.2-amd64 -> 5.8.10-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-amd64-unsigned -> 5.9.6-1~bpo10+1 with priority 500
    firmware-iwlwifi -> 20200918-1~bpo10+1 with priority 500
    firmware-realtek -> 20200918-1~bpo10+1 with priority 500
    firmware-ivtv -> 20200918-1~bpo10+1 with priority 500
    linux-headers-5.8.0-0.bpo.2-amd64 -> 5.8.10-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-rt-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    firmware-qcom-media -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-cloud-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-image-cloud-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-image-amd64-signed-template -> 5.9.6-1~bpo10+1 with priority 500
    firmware-ti-connectivity -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-rt-amd64-dbg -> 5.9.6-1~bpo10+1 with priority 500
    firmware-amd-graphics -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.8.0-0.bpo.2-cloud-amd64-unsigned -> 5.8.10-1~bpo10+1 with priority 500
    linux-headers-rt-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    firmware-myricom -> 20200918-1~bpo10+1 with priority 500
    firmware-samsung -> 20200918-1~bpo10+1 with priority 500
    linux-image-rt-amd64-dbg -> 5.9.6-1~bpo10+1 with priority 500
    firmware-atheros -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.8.0-0.bpo.2-amd64-dbg -> 5.8.10-1~bpo10+1 with priority 500
    firmware-libertas -> 20200918-1~bpo10+1 with priority 500
    firmware-netxen -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.8.0-0.bpo.2-amd64-unsigned -> 5.8.10-1~bpo10+1 with priority 500
    firmware-intelwimax -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-headers-5.9.0-0.bpo.2-common -> 5.9.6-1~bpo10+1 with priority 500
    firmware-linux-nonfree -> 20200918-1~bpo10+1 with priority 500
    firmware-brcm80211 -> 20200918-1~bpo10+1 with priority 500
    firmware-intel-sound -> 20200918-1~bpo10+1 with priority 500
    firmware-cavium -> 20200918-1~bpo10+1 with priority 500
    linux-image-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-headers-5.9.0-0.bpo.2-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-headers-5.8.0-0.bpo.2-cloud-amd64 -> 5.8.10-1~bpo10+1 with priority 500
    firmware-qlogic -> 20200918-1~bpo10+1 with priority 500
    linux-headers-5.8.0-0.bpo.2-common -> 5.8.10-1~bpo10+1 with priority 500
    linux-headers-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-image-cloud-amd64-dbg -> 5.9.6-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-cloud-amd64-unsigned -> 5.9.6-1~bpo10+1 with priority 500
    firmware-misc-nonfree -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-rt-amd64-unsigned -> 5.9.6-1~bpo10+1 with priority 500
    firmware-netronome -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-amd64-dbg -> 5.9.6-1~bpo10+1 with priority 500
    firmware-bnx2x -> 20200918-1~bpo10+1 with priority 500
    linux-image-amd64-dbg -> 5.9.6-1~bpo10+1 with priority 500
    linux-headers-5.9.0-0.bpo.2-cloud-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-headers-5.9.0-0.bpo.2-rt-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    linux-image-rt-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    firmware-qcom-soc -> 20200918-1~bpo10+1 with priority 500
    firmware-linux -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.8.0-0.bpo.2-cloud-amd64 -> 5.8.10-1~bpo10+1 with priority 500
    firmware-bnx2 -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.9.0-0.bpo.2-cloud-amd64-dbg -> 5.9.6-1~bpo10+1 with priority 500
    firmware-ipw2x00 -> 20200918-1~bpo10+1 with priority 500
    linux-image-5.8.0-0.bpo.2-cloud-amd64-dbg -> 5.8.10-1~bpo10+1 with priority 500
    linux-headers-5.9.0-0.bpo.2-common-rt -> 5.9.6-1~bpo10+1 with priority 500
    linux-headers-cloud-amd64 -> 5.9.6-1~bpo10+1 with priority 500
    firmware-siano -> 20200918-1~bpo10+1 with priority 500
    root@omvcopy:/etc/apt/preferences.d#[/tt]

  • Hiya dleidert,


    Many thanks for this. Decided to try it out on my aging OMV3system (Upgrading from OMV3 to OMV4), however it threw a bunch of errors, the final one being....



    There's a bunch of Jessee backports also not existing (yes i know the system is old!)...

    Code
    W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages  404  Not Found
    
    
    W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/contrib/binary-amd64/Packages  404  Not Found
    
    
    W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/non-free/binary-amd64/Packages  404  Not Found
    
    
    E: Some index files failed to download. They have been ignored, or old ones used instead.

    Full log attached..omv_release_upgrade_to_stretch.2021-01-06-092909.txt


    Any pointers appreciated - Thank you for all your hard work on this!!


    Cheers

    Paul

  • Hi - tried to provide all the requested information.



    Code
    sudo apt-cache policy openmediavault
    openmediavault:
    Installed: 4.1.36-1
    Candidate: 4.1.36-1
    Version table:
    *** 4.1.36-1 995
    995 http://packages.openmediavault.org/public arrakis/main amd64 Packages
    995 https://openmediavault.github.io/packages arrakis/main amd64 Packages
    100 /var/lib/dpkg/status

    Hi Alen. Thanks. The script pre.d/72-apt-update-omv-source-list actually overwrites the file /etc/apt/sources.list.d/openmediavault.list with the repository sources for OMV5. In your case it seems this file has been restored to its original content. Now this could have happened when a later script post.d/10-reinstall-omvextras re-installed omv-extras. You say you have it in a VM. I'd really appreciate if you could

    • start a VM with OMV4
    • get the upgrade script
    • in run.d/80-apt-dist-upgrade change the last line which reads exit 0 to exit 1
    • run the upgrade script as before

    It will now exit after the 80-apt-dist-upgrade script has been run. Now please again run:


    cat /etc/apt/sources.list.d/openmediavault.list

    sudo apt-cache policy openmediavault

    sudo apt-get dist-upgrade -u -o Debug::pkgProblemResolver=true --assume-no openmediavault+


    I really appreciate your help.

  • There's a bunch of Jessee backports also not existing (yes i know the system is old!)...

    Code
    W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages  404  Not Found
    
    
    W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/contrib/binary-amd64/Packages  404  Not Found
    
    
    W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/non-free/binary-amd64/Packages  404  Not Found
    
    
    E: Some index files failed to download. They have been ignored, or old ones used instead.

    Let's start with this one. There are no resources for jessie-backports anymore. You can and should disable these. This can be done either via Web GUI (OMV Extras entry -> disable backports) or you comment out the line(s) in /etc/apt/sources.list.d/openmediavault-kernel-backports.list.


    Your log shows that you are not up-to-date with your system. The upgrade scripts (here pre.d/40-update-omv) first tries to bring the system up-to-date before an upgrade is attempted (this is considered "best practices"). In your case your system hasn't seen any updates in quite some time. However apt-get doesn't attempt the update because not all packages could be authenticated. Because you actually have the latest debian-archive-keyring[tt] package installed this might be due to apt not being able to update all sources (the errors above).

    So here are my suggestion for you how to proceed:

    •  disable the backport sources as described above; a [tt]sudo apt-get update should not show any errors; then continue
    • now bring your system up-to-date, either via Web GUI (Update -Management -> <Check>; then mark all packages and <Upgrade>) or run the script in my script folder sudo pre.d/40-update-omv (that's the script that failed)

    If that worked please remove the openmediavault-minidlna and openmediavault-omvextrasorg packages (sudo apt-get remove openmediavault-minidlna openmediavault-omvextrasorg). You can reinstall them after the upgrade succeeded. My script doesn't handle this at the moment for OMV3.


    If that's done too restart the upgrade script. Hopefully it will run through this time.

  • Hi Dleidert


    No problem. Find below my findings


    cat /etc/apt/sources.list.d/openmediavault.list

    sudo apt-cache policy openmediavault

    sudo apt-get dist-upgrade -u -o Debug::pkgProblemResolver=true --assume-no openmediavault+


    I really appreciate your help.


    Code
    root@omvcopy:~/openmediavault-upgrade-4.3# sudo apt-cache policy openmediavault
    openmediavault:
    Installed: 4.1.36-1
    Candidate: 5.5.21-1
    Version table:
    5.5.21-1 500
    500 http://packages.openmediavault.org/public usul/main amd64 Packages
    *** 4.1.36-1 100
    100 /var/lib/dpkg/status
    root@omvcopy:~/openmediavault-upgrade-4.3#


  • If that worked please remove the openmediavault-minidlna and openmediavault-omvextrasorg packages (sudo apt-get remove openmediavault-minidlna openmediavault-omvextrasorg). You can reinstall them after the upgrade succeeded. My script doesn't handle this at the moment for OMV3.


    If that's done too restart the upgrade script. Hopefully it will run through this time.

    Ok thank you Dleidert - after i removed the backports I followed through and it seemed to go swimmingly. Rebooted and all rebooted ok...


    Now on 4.1.36.1..... Now to upgrade to V5x!!!


    Thank you for the ease and relative painfree upgrade - really appreciated!! :)

  • Code
    root@omvcopy:~/openmediavault-upgrade-4.3# sudo apt-cache policy openmediavault
    openmediavault:
    Installed: 4.1.36-1
    Candidate: 5.5.21-1
    Version table:
    5.5.21-1 500
    500 http://packages.openmediavault.org/public usul/main amd64 Packages
    *** 4.1.36-1 100
    100 /var/lib/dpkg/status
    root@omvcopy:~/openmediavault-upgrade-4.3#

    That looks ok.


    So that's interesting. netplan.io is not considered to be installed. So let's find out why:


    sudo apt-cache policy netplan.io

    sudo apt-get install -u -d -o Debug::pkgProblemResolver=true netplan.io (-d will only download the package)


    and/or continue with the dependencies listed as uninstallable.


  • does this help/explain why it's not installed?


    Code
    root@omvcopy:~/openmediavault-upgrade-4.3# sudo apt-cache policy netplan.io
    netplan.io:
    Installed: (none)
    Candidate: 0.95-2
    Version table:
    0.95-2 990
    990 https://deb.debian.org/debian buster/main amd64 Packages
    root@omvcopy:~/openmediavault-upgrade-4.3# ^C
    root@omvcopy:~/openmediavault-upgrade-4.3#
  • Alen , nope. Can you run the installation command (the second command) as well?


    sudo apt-get install -u -d -o Debug::pkgProblemResolver=true netplan.io

    sure. here we go:


  • Ok. netplan.io installs fine. So the dependency resolver gave us the wrong clue. Can you try the following:


    sudo apt-get dist-upgrade -y --no-install-recommends --no-install-suggests -u -o Dpkg::Options::=--force-confdef -o DPkg::Options::=--force-confold" openmediavault+


    If it fails and/or lists a package as uninstallable try installing the package like you did with netplan.io:


    sudo apt-get install -y --no-install-recommends --no-install-suggests -u -o Dpkg::Options::=--force-confdef -o DPkg::Options::=--force-confold" <package>


    Go one package at a time. If that worked too start again with the first command. If that fails or reports uninstallable package please report back to me with the message. We are trying to narrow the culprit down now. I did recreate your system (the package you have including the versions you have) in a VM and it updates fine.

  • mhmm, ok. I'v installed all missing packages. But still geting this issue when trying to upgrade openmediavault+


Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!