Upgrade Scripts for non-interactive major release upgrades (2->3, 3->4, 4->5)

pierre2000 · 26. Januar 2021

Zitat von dleidert

You don't understand what I'm telling you. I gave you a link to a thread with similar error messages you observe. You were supposed to check this thread, and if the suggestions there apply and fix your issues, run the scripts in the post.d/ directory again to complete the upgrade.

I did exactly what you suggested. I am sorry if I gave you the impression that I did NOT follow them. And, right, I am just a lousy old user, who has half the time no clue, sorry for my lack of knowledge, thats why I am looking for help, i am just too stupid to get problems like this solved on my own. Give me a f..x..ed car, machine, aeroplane, broken cupboard or leaking plumbing, THERE I can help, here I need help.

I did what was done in the thread you gave the link, which was, btw, without any error message, run your scripts again without success, or, the same errors as before. And, as written in#115, you were definitely right, the upgrade was not complete.

z0rk · 2. Februar 2021

dleidert This is awesome! I have three systems all running 4.x. Only one of them uses an encrypted storage device. The boot devices for all three are USB keys. Are there any special considerations to be given? You mentioned a caveat re: encrypted drives but I am not sure I caught the specifics. Presumably if my boot devices have enough free space available it shouldn't make any difference that they are USB keys.

ryecoaaron Advises to uninstall all plug-ins when upgrading from 4.x to 5.x. Should I do this and after the upgrade re-install them will my plug-in configuration settings be lost?

Thanks for the great work!

ryecoaaron · 2. Februar 2021

Zitat von z0rk

Should I do this and after the upgrade re-install them will my plug-in configuration settings be lost?

I you don't purge the plugin (use remove), then the settings aren't lost. I'm not sure what the upgrade scripts are doing now.

dleidert · 2. Februar 2021

Zitat von z0rk

I have three systems all running 4.x. Only one of them uses an encrypted storage device. The boot devices for all three are USB keys. Are there any special considerations to be given? You mentioned a caveat re: encrypted drives but I am not sure I caught the specifics. Presumably if my boot devices have enough free space available it shouldn't make any difference that they are USB keys.
ryecoaaron Advises to uninstall all plug-ins when upgrading from 4.x to 5.x. Should I do this and after the upgrade re-install them will my plug-in configuration settings be lost?

Let's split this into three parts:

USB boot devices: Are you using some kind of chainloading? Or what are the specifics here? Important IMHO is to update grub at the boot device too especially if it is chain-loading. In this case the system upgrade will only update the grub MBR of the device holding the operating system. I left some hints in this first post of this thread about a general issue with grub.

Encrypted disks: You shoud read the Buster release notes as suggest by the first post too and check if you are affected by any of the changes listed there. If you have just a common setup you probably won't have any issues to expect.

Plugins: The scripts will remove the plugins not compatible with OMV 5. So this step is not mandatory.

z0rk · 3. Februar 2021

Zitat von ryecoaaron

I you don't purge the plugin (use remove), then the settings aren't lost. I'm not sure what the upgrade scripts are doing now.

Excellent, thanks.

z0rk · 3. Februar 2021

Zitat von dleidert

Let's split this into three parts:

USB boot devices: Are you using some kind of chainloading? Or what are the specifics here? Important IMHO is to update grub at the boot device too especially if it is chain-loading. In this case the system upgrade will only update the grub MBR of the device holding the operating system. I left some hints in this first post of this thread about a general issue with grub.

Encrypted disks: You shoud read the Buster release notes as suggest by the first post too and check if you are affected by any of the changes listed there. If you have just a common setup you probably won't have any issues to expect.

Plugins: The scripts will remove the plugins not compatible with OMV 5. So this step is not mandatory.

Alles anzeigen

No, I am not chainloading. I have three OMV servers. All of them boot from one (1) USB key. All just plain old vanilla installations as per OMV documentation.

Thanks, I will check out the Buster release notes.

Ok, remove plugins but not purge correct?

Thank you, Sir.

pbertra · 3. Februar 2021

I did the upgrade and I ended up with the following errorin the GUI:

Error #0:

OMV\Exception: Failed to open include file 'js/omv/PluginManager.js'. in /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc:42
Stack trace:
#0 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(70): OMV\ControlPanel\ControlPanelAbstract->scanFiles('js/omv/PluginMa...', Array)
#1 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(87): OMV\ControlPanel\ControlPanelAbstract->scanFiles('js/omv/PluginMa...', Array)
#2 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(90): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#3 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(90): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#4 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(90): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#5 /usr/share/php/openmediavault/controlpanel/administration.inc(54): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#6 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(208): OMV\ControlPanel\Administration->getJavascriptIncludes()
#7 /var/www/openmediavault/index.php(41): OMV\ControlPanel\ControlPanelAbstract->render()
#8 {main}

The upgrade script finished with :

Errors were encountered while processing:

openmediavault-omvextrasorg

E: Sub-process /usr/bin/dpkg returned an error code (1)

run-parts: /root/openmediavault-upgrade-4.7/post.d/87-apt-autoremove-dpkg-rc exited with return code 123

Moreover, plex fails to start.

dleidert · 3. Februar 2021

Zitat von pbertra

I did the upgrade and I ended up with the following errorin the GUI:
Error #0:

OMV\Exception: Failed to open include file 'js/omv/PluginManager.js'. in /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc:42
Stack trace:
#0 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(70): OMV\ControlPanel\ControlPanelAbstract->scanFiles('js/omv/PluginMa...', Array)
#1 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(87): OMV\ControlPanel\ControlPanelAbstract->scanFiles('js/omv/PluginMa...', Array)
#2 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(90): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#3 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(90): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#4 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(90): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#5 /usr/share/php/openmediavault/controlpanel/administration.inc(54): OMV\ControlPanel\ControlPanelAbstract->scanDir('js/omv/module/a...', Array, true)
#6 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(208): OMV\ControlPanel\Administration->getJavascriptIncludes()
#7 /var/www/openmediavault/index.php(41): OMV\ControlPanel\ControlPanelAbstract->render()
#8 {main}

The upgrade script finished with :
Errors were encountered while processing:
openmediavault-omvextrasorg
E: Sub-process /usr/bin/dpkg returned an error code (1)
run-parts: /root/openmediavault-upgrade-4.7/post.d/87-apt-autoremove-dpkg-rc exited with return code 123

Moreover, plex fails to start.

Alles anzeigen

What's the output of dpkg -l | grep openm? Also you need to provide the upgrade log file. Otherwise I cannot check what might have been gone wrong.

huebnerd · 4. Februar 2021

At first: many thanks for you work!

I did the upgrade from OMV4 to OMV5 on an simple raspberrypi 4b with one usb-hd booting from sd-card.

Most of the upgrade works - but at "install apt-show-versions" exits the script - have a lokk at the attachment.

As beginner I'm now unsure what to do.

omv_release_upgrade_to_buster.2021-02-04-101256.txt

dleidert · 4. Februar 2021

Zitat von huebnerd

I did the upgrade from OMV4 to OMV5 on an simple raspberrypi 4b with one usb-hd booting from sd-card.
Most of the upgrade works - but at "install apt-show-versions" exits the script - have a lokk at the attachment.
As beginner I'm now unsure what to do.

omv_release_upgrade_to_buster.2021-02-04-101256.txt

Hm. I've never seen it. But I quick search did revealed this bug report and a possible solution. Try

sudo apt-get -o Acquire::GzipIndexes=false update && sudo apt-get install -f

Maybe you have to purge apt-show-versions first though.

Afterwards you need to run all the remaining scripts in post.d/:

. inc/envvars

mv post.d/10-reinstall-omvextras post.d/10-reinstall-omvextras.disabled (because this has ran successfully)

sudo run-parts -v --exit-on-error post.d (will run everything else)

I'll have to work on a solution for that. It is clearly a bug in apt-show-versions.

pbertra · 4. Februar 2021

It did work in the end at the third try.

I cleaned up all obsolete packages, manually removed the plex plugin, cleaned my apt sources...
I guess my system was not clean enough. It was already upgraded from 3 to 4...

Thank you for your nice work

z0rk · 5. Februar 2021

omv_release_upgrade_to_buster.2021-02-04-151446.log.txt

I am upgrading from 4 to 5. I've disabled all services, but did not remove any plug-ins. I am not sure what the error is other than something possibly related to 'NoneType' and the py?

Thanks

dleidert · 5. Februar 2021

Zitat von z0rk

I am upgrading from 4 to 5. I've disabled all services, but did not remove any plug-ins. I am not sure what the error is other than something possibly related to 'NoneType' and the py?

The python error is a well known bug in /usr/lib/python3.5/weakref.py and not related to the scripts exiting.

Your /srv/dev-disk-by-label-muZak_bk mount point cannot be remounted because the remount operation is (for some reason I don't know) not supported. So with the error you are told that instead of the remount operation one would have to unmount and mount this mount point again. Now the pre.d/20-remount-ro script is currently mandatory. So either you find out why this mount point doesn't support remounting. Then we might find a way to work around it. Or: You can unmount this mountpoint and mount it read-only to protect the data on it. Then rename pre.d/20-remount-ro to pre.d/20-remount-ro.disabled and run the omv-release-upgrade-5 script again.

z0rk · 5. Februar 2021

I went with the un-mount and re-mount read-only option. Now for the past hour the process has been stuck at:

Configuration file '/etc/collectd/collectd.conf'

==> Modified (by you or by a script) since installation.

==> Package distributor has shipped an updated version.

==> Keeping old config file as default.

Setting up tasksel-data (3.53) ...

Setting up tasksel (3.53) ...

Setting up libdevmapper1.02.1:amd64 (2:1.02.155-3) ...

Setting up libdevmapper-event1.02.1:amd64 (2:1.02.155-3) ...

Setting up dmsetup (2:1.02.155-3) ...

update-initramfs: deferring update (trigger activated)

Setting up lvm2 (2.03.02-3) ...

Installing new version of config file /etc/lvm/lvm.conf ...

When I hit the Enter key the cursor moves one line down but nothing else happens.

I am able to open another SSH session though.

huebnerd · 5. Februar 2021

Zitat von dleidert

Hm. I've never seen it. But I quick search did revealed this bug report and a possible solution. Try

sudo apt-get -o Acquire::GzipIndexes=false update && sudo apt-get install -f

Maybe you have to purge apt-show-versions first though.

Afterwards you need to run all the remaining scripts in post.d/:

./ inc/envvars
mv post.d/10-reinstall-omvextras post.d/10-reinstall-omvextras.disabled (because this has ran successfully)
sudo run-parts -v --exit-on-error post.d (will run everything else)

I'll have to work on a solution for that. It is clearly a bug in apt-show-versions.

Alles anzeigen

I will try this a soon as possible - thanks for the detailed help and the fast response!

dleidert · 5. Februar 2021

Zitat von z0rk

Now for the past hour the process has been stuck at:

Configuration file '/etc/collectd/collectd.conf'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
==> Keeping old config file as default.
Setting up tasksel-data (3.53) ...
Setting up tasksel (3.53) ...
Setting up libdevmapper1.02.1:amd64 (2:1.02.155-3) ...
Setting up libdevmapper-event1.02.1:amd64 (2:1.02.155-3) ...
Setting up dmsetup (2:1.02.155-3) ...
update-initramfs: deferring update (trigger activated)
Setting up lvm2 (2.03.02-3) ...
Installing new version of config file /etc/lvm/lvm.conf ...

Alles anzeigen

Find out what's hanging: ps -ef --forest

dleidert · 5. Februar 2021

Zitat von huebnerd

I will try this a soon as possible - thanks for the detailed help and the fast response!

There was a bug in my commands: Run:

. inc/envvars (no slash after the dot)

z0rk · 5. Februar 2021

Zitat von dleidert

Find out what's hanging: ps -ef --forest

I am not sure I was able to glean much more from ps, but an installation log was generated.

ps_ef_forest.txt and omv_release_upgrade_to_buster.2021-02-04-222627.log.txt

thanks, DL

dleidert · 5. Februar 2021

Zitat von z0rk

I am not sure I was able to glean much more from ps, but an installation log was generated.

ps_ef_forest.txt and omv_release_upgrade_to_buster.2021-02-04-222627.log.txt

Here is the relevant part:

Code

SCREEN -dmS mysession
 \_ /bin/bash
     \_ sudo ./omv-release-upgrade-5
         \_ /bin/sh ./omv-release-upgrade-5
             \_ /bin/sh ./omv-release-upgrade-5
             |   \_ run-parts -v --exit-on-error /tmp_update/openmediavault-upgrade-4.7/run.d
             |       \_ /bin/sh /tmp_update/openmediavault-upgrade-4.7/run.d/50-apt-upgrade
             |           \_ apt-get upgrade -y --no-install-recommends --no-install-suggests -u -o Dpkg::Options::=--force-confdef -o DPkg::Options::=--force-confold collectd+ rrdcached+
             |               \_ /usr/bin/dpkg --force-confdef --force-confold --status-fd 45 --configure --pending
             |                   \_ /bin/sh /var/lib/dpkg/info/lvm2.postinst configure 2.02.168-2
             |                       \_ vgcfgbackup
             \_ tee omv_release_upgrade_to_buster.2021-02-04-222627.log

Alles anzeigen

So it hangs executing vgcfgbackup. Sounds like these #439063, LP#502695. You can try to kill vgcfgbackup which in theory should continue the upgrade process but will kill the lvm2.postinst script too. If I would go this way I would probably run:

sudo apt-get install --reinstall lvm2

after the upgrade process finishes to make sure the postinst script had run all tasks. If that again hangs you can edit /var/lib/dpkg/info/lvm2.postinst and remove the the call to the backup script and run

sudo apt-get install -f

It would actually be a good idea to find out what is blocking the backup script.

dleidert · 5. Februar 2021

JFTR. I'm currently working on improving the scripts so if one fails the user can fix an occurring issue and then continue at the point the script left.

Jetzt mitmachen!