Upgrade Scripts for non-interactive major release upgrades (2->3, 3->4, 4->5)

Alen, I just noticed that #33 doesn't show the full resolver log, only the last lines.

Quote from Alen

mhmm, ok. I'v installed all missing packages. But still geting this issue when trying to upgrade openmediavault+

Code

root@omvcopy:~/openmediavault-upgrade-4.3# sudo apt-get dist-upgrade -y --no-install-recommends --no-install-suggests -u -o Dpkg::Options::=--force-confdef -o DPkg::Options::=--force-confold openmediavault+
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Error!
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:


The following packages have unmet dependencies:
openmediavault : Depends: salt-minion (>= 3000.2) but 2018.3.4+dfsg1-6+deb10u1 is to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
root@omvcopy:~/openmediavault-upgrade-4.3#

Display More

Ok. salt-minion seems to be the culprit now. The version to be installed is the one from Debian, not the one from omv. So what do these commands say:

sudo apt-cache policy salt-minion

sudo apt-get install -u -d -o Debug::pkgProblemResolver=true salt-minion

sudo apt-get install -u -d -o Debug::pkgProblemResolver=true salt-minion=3002.2+ds-1

Quote from Alen

that's the output from all 3 commands:

Code

root@omvcopy:~# sudo apt-get install -u -d -o Debug::pkgProblemResolver=true salt-minion=3002.2+ds-1
Reading package lists... Done
Building dependency tree
Reading state information... Done
Starting pkgProblemResolver with broken count: 1
Starting 2 pkgProblemResolver with broken count: 1
Investigating (0) salt-minion:amd64 < 2018.3.4+dfsg1-6+deb10u1 -> 3002.2+ds-1 @ii pumU Ib >
Broken salt-minion:amd64 Depends on salt-common:amd64 < 2018.3.4+dfsg1-6+deb10u1 @ii mK > (= 3002.2+ds-1)
Considering salt-common:amd64 0 as a solution to salt-minion:amd64 9999
Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:


The following packages have unmet dependencies:
salt-minion : Depends: salt-common (= 3002.2+ds-1) but 2018.3.4+dfsg1-6+deb10u1 is to be installed
E: Unable to correct problems, you have held broken packages.
root@omvcopy:~#

Display More

Yes, that should be

sudo apt-get install -u -d -o Debug::pkgProblemResolver=true salt-minion=3002.2+ds-1 salt-common=3002.2+ds-1

then to succeed. However the interesting part is that on my system the packages from packages.openmediavault.org have a higher priority (995) then the ones from Debian (990). While on your system for salt* it shows the the standard priority of 500. Compare it to my test system:

$ sudo apt-cache policy salt-minion
salt-minion:
  Installed: 2018.3.4+dfsg1-6+deb10u1
  Candidate: 3002.2+ds-1
  Version table:
     3002.2+ds-1 995
        995 http://packages.openmediavault.org/public usul/main amd64 Packages
 *** 2018.3.4+dfsg1-6+deb10u1 990
        990 https://deb.debian.org/debian buster/main amd64 Packages
        990 https://deb.debian.org/debian-security buster/updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2016.11.2+ds-1+deb9u6 500
        500 http://deb.debian.org/debian-security stretch/updates/main amd64 Packages
     2016.11.2+ds-1+deb9u4 500
        500 http://deb.debian.org/debian stretch/main amd64 Packages
     2014.1.13+ds-3+deb8u1 500
        500 http://security.debian.org/debian-security jessie/updates/main amd64 Packages

$ cat /etc/apt/preferences.d/openmediavault.pref
Package: *
Pin: origin packages.openmediavault.org
Pin-Priority: 995

It seems that /etc/apt/preferences.d/openmediavault.pref on my test system is a remnant of an earlier openmediavault version and sets the priority high enough so it works. You didn't list this file though when I asked for the contents of /etc/apt/preferences.d/. But in one of your earlier replies the output of sudo apt-cache policy was:

Quote

995 https://openmediavault.github.io/packages arrakis/main amd64 Packages

release o=openmediavault.org archive,a=arrakis,n=arrakis,l=openmediavault.org archive,c=main,b=amd64
origin openmediavault.github.io

995 http://packages.openmediavault.org/public arrakis/main amd64 Packages
release o=openmediavault.org archive,a=arrakis,n=arrakis,l=openmediavault.org archive,c=main,b=amd64
origin packages.openmediavault.org

So there must be some pinning somewhere. Would you be so kind to post the contents of the files found in /etc/apt/preferences.d/?

I think that's all I need to reproduce the issue now.

toibs because there might be an issue that omv4 is not upgraded please check what the following says:

sudo apt-cache policy openmediavault

And as always I'd appreciate the log file.

Alen the issue has been fixed in release 4.4 of the script. You should now be able to upgrade.

You ran into the same issue as Alen did.

Quote from toibs

root@openmediavault:/# sudo apt-cache policy openmediavault

openmediavault:

Installed: 4.1.36-1

Candidate: 5.5.21-1

Version table:

5.5.21-1 500

500 http://packages.openmediavault.org/public usul/main amd64 Packages

*** 4.1.36-1 100

100 /var/lib/dpkg/status

Display More

Parts of your system have already been upgraded. OMV has not yet been upgraded. We are going to fix it by hand. Get the latest source

wget -qO- https://github.com/dleidert/openmediavault-upgrade/archive/v4.4.tar.gz | tar -xz
cd openmediavault-upgrade-4.4

and instead of running the whole script, run the remaining scripts manually.:

. inc/envvars
sudo pre.d/01-create-omvvars
sudo pre.d/72-apt-update-omv-source-list
sudo pre.d/73-apt-source-remove-docker
sudo pre.d/75-apt-source-armbian-fix
sudo run.d/10-apt-update
sudo run.d/80-apt-dist-upgrade
sudo run.d/81-apt-dist-upgrade-omv

Now after having done this sudo apt-cache policy openmediavault should show the 5.x version installed. Then run the remaining tasks:

sudo run-parts -v --exit-on-error post.d

Now everything should be fine.

Quote from toibs

sudo run.d/80-apt-dist-upgrade - this comes up clear (all up to date)

However sudo run.d/81-apt-dist-upgrade-omv brings up the following :

Code

root@openmediavault:/openmediavault-upgrade-4.4# sudo run.d/81-apt-dist-upgrade-omv
run.d/81-apt-dist-upgrade-omv: 6: .: Can't open /openmediavault-upgrade-4.4/run.d/../inc/omvvars

Sorry

Yes. Run

sudo pre.d/01-create-omvvars

first. It will create the missing file inc/omvvars.

Quote from vcdwelt

Now i will also try to Update my OMV4 to OMV5.

Just to be sure, because was often running into problems on Upgrades:

- I need to delete the Extra Repo

- Also i need to uninstall all Plugins?

- Still using Plex as "Plugin" because had bad experince with Plex as a Docker and i don't really want to move my Plex into a Docker. Is it possible to keep Plex installed during the Update process?

Backup:

Did a Backup via the Backup Plugin (fsarchiver), so all should be save, if i run into problems and i can restore the complete OMV installation as it was at the time of the Backup?

Sorry for this Questions, but i was reading the half day during the Forum and wasn't 100% sure about this topics. Don't want to crash my great running OMV installation *scarry*

Display More

The scripts currently deal with omvextras installed. And they will remove all plugins not available for OMV5.

Is plex from one of the repositories omvextras adds to its menu? Then you should be fine. Do you have any third-party software installed, either not being installed in Debian package format or pulled in from any non-omv resource (not counting the resources omvextras enables)? And the scripts are untested for Raspbian users.

It is good you have a backup. Then you should be able to replay it to the system. To restore the system's filesystem one has to use a live-cd or extra system though. There was a thread recently how to play such a backup back. However I haven't seen a non-salvageable system in my time.

I understand the feelings. I can test it in a VM if you want. You need to provide me the package list

dpkg-query -W -f='${binary:Package}=${Version}\n' | sort > package.list

and which repositories from omvextras you have enabled and if you have the backports repository enabled.

Quote from Alen

Do you still need to content of /etc/apt/preferences.d/ for further analysis?

No, thanks.

Quote from Alen

I also had Plex/Duplicati/Rsync installed pre-upgrade - there are all gone now. Also Dockers is gone with all my dockers installed. This is more of a problem. Is this expected behaviour?

There is a list of plugins not available for omv5. These plugins get removed before attempting the upgrade. The plugin list includes openmediavault-docker-gui, openmediavault-duplicati, and openmediavault-shellinabox in your case. At the moment the script pre.d/50-remove-plugins removes these plugins as well es the packages they pulled in completely. It might not be ideal, but it is the cleanest solution at the moment. I can make a more "defensive" approach and just remove the plugin package, leaving the other packages untouched. That may lead to systems having a lot of cruft installed afterwards.

I'm open for suggestions how it should work.

I had thought about the same solution. I've added a script to mark a defined list of packages as manually installed. The list currently consists of the three packages you mention. If there are more it can be extended. Will be part of release 4.5.