____________________________________________________________________________________________________

EDIT:

After much testing it turned out to be necessary to perform some more actions for this to work. So I developed this app that does everything automatically.

Install: wget -O - https://raw.githubusercontent.com/xhente/omv-regen/master/omv-regen.sh | bash

Thread

[How to] Migrate/Restore OMV Settings to Another System with omv-regen

UTILITY TO MIGRATE OMV CONFIGURATION TO ANOTHER SYSTEM OR
DO A NEW OMV INSTALLATION AND KEEP THE SETTINGS

omv-regen

Copy and paste the following line into a terminal and run it as root or with sudo.
sudo wget -O - https://raw.githubusercontent.com/xhente/omv-regen/master/omv-regen.sh | sudo bash

• STEP 1. Create a backup of the original system with omv-regen.
• STEP 2. Do a fresh installation of OMV on the disk you want and connect the original data drives.
• STEP 3. Use omv-regen to migrate the

…

chente

Apr 29th 2023

____________________________________________________________________________________________________

Some time ago someone known to everyone on this forum said:

Quote

It would be interesting to re-architect OMV plugins to be saltstack (or better yet - ansible ) code instead of .deb packages. Then you could run the "playbook" and it would install the packages and configure everything. That would make things easier.

It seems that this moment has arrived, OMV6 has already been working with salt for a year. So it occurred to me that thanks to salt maybe a complete OMV system can be recreated from scratch from the database xml file. I have put my hands to work and it has worked for me. I have managed to recreate an existing system from scratch with ease.

The procedure is simple. In short it consists of doing a fresh installation (update, install plugins, do not configure anything, just install) add existing users, replace the config.xml file, run two commands and reboot.

When the system boots it will be in the same state as we had it, all the GUI settings we had, file system, permissions, shared folders, services, scheduled jobs, rsync, plugin settings, etc will be in their original state.

Exceptions and points to consider:

• It is recommended to have the system as up-to-date as possible before extracting config.xml to copy it to the new system. Otherwise, the versions of some plugins may change and some inconsistencies may appear.
• All manual root settings outside of the GUI will be gone, as expected. If you have done this you already know what you have to do to recreate it.
• If you have shared folders in root, they will logically disappear. It is not advised to do this in general anyway.
• Podman based plugins (Filebrowser, Photoprism, Wetty) will be active and configured in the GUI. But the internal configurations of the container will have been lost.
• It is necessary to create the users again, they are not in the database. They're in /etc/passwd and the (encrypted) passwords are in /etc/shadow, but you'll end up doing it by hand first.
• The docker containers will be configured in openmediavault-compose, you only have to start them the first time. If you installed them with Portainer you will have to reinstall them.
• Some special plugins might lose settings. For example Openmediavault-KVM will lose the configuration of the vm. It can be fixed with backup copies of the xml files. If you had the proxmox kernel you must previously install it with the plugin.I don't know all the plugins so I can't make an exhaustive list here. But in general all will work except for some exceptions.

I think this could be used for an OMV plugin. It is only necessary to follow the working route. As I see it, a button in the GUI sets a different folder on disk to do the backup (config.xml file, user list and plugin list). Once the system has been reinstalled by the user, there is a button to regenerate the system, you only have to say in which folder the backup is. The system installs the plugins, generates the users and resets the database. It reboots and voila, new system.

As an example, in the following post is one of the tests I have done.

REGENERATION OF AN OMV SYSTEM

ORIGINAL SYSTEM:

Original system general configuration

hostname -> testomv

Standard configuration -> Time zone, Notification, Power management, Certificates, Scheduled Tasks .....

Network -> br0 bridge created with fixed IP

Filesystem -> 1 disk OS and 2 disks Raid1 BTRFS

Smart active with tests scheduled

2 shared folder on Raid1 BTRFS -> testfolder1 and testfolder2

2 users -> testuser1 and testuser2 -> write permissions on testfolder1 and testfolder2 respectively

Samba -> shared folders testfolder1 and testfolder2 working

Rsync -> scheduled task copy from testfolder1 to testfolder2

All container volumes are on the data disks.

Plugins configured in the original system

Openmediavault-Filebrowser (podman) -> configured in testfolder1 with certificate and working

Openmediavault-OMV-Extras -> apparmor patch applied, docker installed and working

Openmediavault-Compose -> Handbrake and Jellyfin installed and working

Openmediavault-Symlinks -> 1 symlink created

Openmediavault-Backup -> configured backup and scheduled task

Openmediavault-Wetty (podman) -> configured and working

DATA NEEDED TO REGENERATE THE SYSTEM

To regenerate this system we need to have the following:

- The OMV database configuration file, which is in /etc/openmediavault/config.xml -> copy to our PC.

- A list of existing users on the system (passwords are not required). If there are many you can list them like this:

# cat /etc/passwd | grep /bin/sh
testuser1:x:1000:100::/home/testuser1:/bin/sh
testuser2:x:1001:100::/home/testuser2:/bin/sh

- A list of installed plugins. We can get it like this:

# dpkg -l | grep openmediavault
ii  openmediavault                    6.3.5-2                       all          openmediavault - The open network attached storage solution
ii  openmediavault-backup             6.1                           all          backup plugin for OpenMediaVault.
ii  openmediavault-compose            6.6.1                         all          OpenMediaVault compose plugin
ii  openmediavault-filebrowser        6.0.4-1                       all          openmediavault File Browser plugin
ii  openmediavault-keyring            1.0                           all          GnuPG archive keys of the OpenMediaVault archive
ii  openmediavault-omvextrasorg       6.1.1                         all          OMV-Extras.org Package Repositories for OpenMediaVault
ii  openmediavault-sharerootfs        6.0.2-1                       all          openmediavault share root filesystem plugin
ii  openmediavault-symlinks           6.0.4                         all          OpenMediaVault symlinks plugin
ii  openmediavault-wetty              6.0.4-1                       all          openmediavault WeTTY (Web + TTY) plugin

SYSTEM REGENERATION

- Do a clean install of OMV. Update the system and connect data disks.

- Install the plugins that the previous system had. You don't need to configure anything at all, just install, apply the apparmor patch if you need it.

- Create the existing users on the original machine, do it in the same order as the UIDs, 1000 first, 1001 second, 1002 third,... the passwords don't need to be the same. Change the default admin password to one of your own choosing.

- Replace the existing /etc/openmediavault/config.xml file with the one we had on the original system and run:

omv-salt stage run prepare

omv-salt stage run deploy

- If you don't want to lose the connection just set the original server IP first. If you haven't just wait 2 minutes and restart.

- Now we should be able to access the OMV GUI on our old IP. We can verify that all the previous system configurations appear.

RESULT CHECK

General configuration of the new system

hostname -> testomv -> OK

Standard configuration -> Time zone, Notification, Power management, Certificates, Scheduled Tasks ..... -> ALL OK

Network -> br0 bridge created with fixed IP -> OK

Filesystem -> 1 disk OS and 2 disks Raid1 BTRFS -> OK

Smart active with tests scheduled -> OK

2 shared folder on Raid1 BTRFS -> testfolder1 and testfolder2 -> OK

2 users -> testuser1 and testuser2 -> write permissions on testfolder1 and testfolder2 respectively -> OK

Samba -> shared folders testfolder1 and testfolder2 working -> OK

Rsync -> scheduled task copy from testfolder1 to testfolder2 -> OK

Plugins configured in the new system

Openmediavault-Filebrowser (podman) -> configured in testfolder1 with certificate and working -> Working and configured in the GUI but the internal configuration of the container has been lost. You could try to backup the pod configuration.

Openmediavault-OMV-Extras -> apparmor patch applied, docker installed and working -> OK

Openmediavault-Compose -> Handbrake and Jellyfin installed and working -> OK You only have to start the containers the first time.

Openmediavault-Symlinks -> 1 symlink created -> OK

Openmediavault-Backup -> configured backup and scheduled task -> OK

Openmediavault-Wetty (podman) -> configured and working -> OK

CONCLUSION

If the precautions of the previous post are taken into account, a broken system could easily be regenerated without the need to make complete backups of the system disk. Just update the system, extract the config.xml file and do the rebuild.

EDIT: The system has evolved. See post 48. RE: How to regenerate a complete OMV system.

Quote from raulfg3

please make this post sticky

Patience, let's see what the wise minds think...

hmm... I'm not sure I understand well. You mean this?

Quote from chente

It is recommended to have the system as up-to-date as possible before extracting config.xml to copy it to the new system. Otherwise, the versions of some plugins may change and some inconsistencies may appear.

Or is there something else?

The idea would be to have the system always as up-to-date as possible. It is what we all try to do in general.

This way you can have a cron job scheduled to backup config.xml and always have it ready to rebuild.

Quote from macom

However, there is also the possibility to install older versions of the plugins on a fresh install, in case you know which one were installed when making the copy of config.xml.

That is interesting. Where can they be downloaded from?

Quote from macom

the OMV version and the plugin version have to be the ones which have been installed when the backup of config.xml was made.

You could add a file with the output of dpkg -l | grep openmediavault to the backup folder where the files are stored. In this way, the versions corresponding to the moment of extraction of config.xml will be known at all times.

Something similar could be done with the users, although this is less important, it is assumed that they do not change as frequently.

Quote from chente

You could add a file with the output of dpkg -l | grep openmediavault to the backup folder where the files are stored. In this way, the versions corresponding to the moment of extraction of config.xml will be known at all times.

Something similar could be done with the users, although this is less important, it is assumed that they do not change as frequently.

Actually I already mentioned this here in the first post. I did not remember it. I think I made it too long even for me

Quote from chente

As I see it, a button in the GUI sets a different folder on disk to do the backup (config.xml file, user list and plugin list).

macom Do you see any other problem in this method? It is safe? I have tested it on new installations. I'm not sure what would happen in older installations.

I already see the problem. Only the latest version of each package is available for installation.

# apt-cache madison openmediavault
openmediavault |    6.3.5-2 | https://packages.openmediavault.org/public shaitan/main amd64 Packages
# apt-cache madison openmediavault-filebrowser
openmediavault-filebrowser |    6.0.4-1 | https://packages.openmediavault.org/public shaitan/main amd64 Packages

I can't find a way to install a package that is not the latest version. Unless there is another way, maybe download it from somewhere and install it manually. But this would complicate things quite a bit.

Therefore, unless someone provides another solution, this method would only work with full guarantees in those cases where the original system is up to date.

In other cases the new system will have higher package versions, and this will lead to possible inconsistencies. Everything will depend on the differences between the packages.

Summarizing. This method is used to rebuild an updated system. This use case can still be very useful. Cases where the system is still operational and can be upgraded. This is very common. Upgrade, extract the database, rebuild the system.

Someone correct me if I'm wrong.

The way I see it, regeneration can be safely applied in the following situations:

• I want to replace the system disk to SSD, or to USB, or to nvma, or ... Or simply because it does not work correctly or because it is too small.
• I want to replace the hardware completely and the original installation is not compatible with the new hardware.
• I want to have a simple OS backup, which also does not take up space and is safe.
• I want to reinstall the system because it has become unstable and the installed packages are up to date.
• Others that haven't occurred to me yet...

These situations are very common in the forum and generate long threads. Regeneration is an effective and efficient solution. votdev I think you should speak on this. "Silence grants, the smile confirms and the look speaks"

Quote from barlowg

If I was not thinking about a complete rebuild of my OMV system (to fix the old kernel version issue from in-place update omv5->omv6 on RPI4), I would be happy to give this a go.

If the system is fully upgraded to OMV6 I think it should work. Make sure you have a backup beforehand. Luck!

Something simple that anyone can use to back up essential files for a rebuild might be the following. You don't need to make a script for this.

Just create a daily scheduled task in the OMV GUI and add this command to run:

rsync -av /etc/openmediavault/config.xml /[PATH_TO_DATA]/regen_omv/ && dpkg -l | grep openmediavault > /[PATH_TO_DATA]/regen_omv/plugins_list && cat /etc/passwd | grep /bin/sh > /[PATH_TO_DATA]/regen_omv/users_list

Replace [PATH_TO_DATA] with the path to the folder where you want to store this information.

This task will make a copy of the database, create a list of installed plugins including their versions, and another list of system users. The information will be updated every day. It's all you need to regenerate the system.

Warning: Store this data securely, the database contains sensitive information from your server.

Edit: There is another more elaborate and complete script later in this thread. RE: How to regenerate a complete OMV system.

If you have more information on the OS disk that you need to keep, this script may be useful.

#!/bin/bash

# Sets the current date
dt=$(date +%y%m%d)

# Sets the path to store the backup.
# Replace PATH_TO_DATA with the path of the folder where you want to store the backup.
bkp="/PATH_TO_DATA/regen_omv/"

# Helpful message if you set notifications on the scheduled task
echo ""
echo "Backup to regenerate the system with date $dt in $bkp"
echo ""

# Does the extraction of the database, the list of users and the list of plugins simultaneously
rsync -av /etc/openmediavault/config.xml ${bkp} && dpkg -l | grep openmediavault > ${bkp}plugins_list && cat /etc/passwd | grep /bin/sh > ${bkp}users_list

# It does the extraction of the existing data in the /home folder.
# Remove the hash mark from the next line if you need to.
#rsync -av --delete /home/ ${bkp}home

# It does the extraction of the xml files of virtual machines and KVM network configurations. 
# Remove the hash mark from the next line if you need to.
#[ -d /etc/libvirt/qemu ] && rsync -av --delete /etc/libvirt/qemu/ ${bkp}kvm

# You can copy and paste the line above and substitute the folder you want to have a backup of. 
# For example, the following would extract the information from a custom installation of wireguard.
# Remove the hash mark from the next line if you need to.
#[ -d /etc/wireguard ] && rsync -av --delete /etc/wireguard/ ${bkp}wireguard

# Generates a file in the backup folder with the extraction date.
[ -f ${bkp}date* ] && rm ${bkp}date*
touch ${bkp}date_$dt

Copy it to a text file and save it with the name regen_omv.sh

Copy this file to the server's /home folder and make it executable with chmod 700 /home/regen_omv.sh

Then schedule a daily task in the OMV GUI that runs the command /home/regen_omv.sh

And that's it, every day the backup data will be updated.