Anyone using rpi-clone to automatically "live" backup the whole Raspi boot micro SD?

  • Hi all,


    I'm successfully using rpi-clone on almost all my Raspi, where I regularly swap boot/clone SD to check the last backup will work in case of boot SD I/O problems. I'm very satisfied with rpi-clone's system backups.


    On OMV, booting the rpi-cloned SD gives problems with the Raspberry PI 2/3/4 OMV distributed image (Armbian), partly because some /var/ subforders are lost maybe in the "rsync --with-root-excludes" process.


    After booting with the cloned SD, JellyFin works regularly, but no OMV web UI. I noticed an error in one openmediavault-*.service, so I manually created the missing /var/log/openmediavault/ and executed omv-initsystem, but still no luck.



    Further hints?



    Thank you,


    Piero

  • Thanks @crashtest that's a very good beginners guide, however it doesn't seem to cover rpi-clone.


    My question isn't about cloning SD cards, but automatically "live" cloning the running OMV armbian SD, I will try to edit the thread title.


    My issue is specifically about rpi-clone, with which I successfully get unattended backup of the whole SD on a dozen of other Raspi servers, and I would like to apply also to OMV.


    Piero

  • Did you find another solution OP? I have had a look at the backup plugin, but it looks like it only creates image files and is not as simple to recover from or to roll back as rpi-clone is. I agree with the OP that it is likely to do with the --with-root-excludes flag that is added by default when one performs a clone.


    From rpi-clone's github:

    I am able to rule out some of the culprits:

    • .gvfs: non-existant in the root of my working raspbian install;
    • /dev/: seems unlikely that OMV is running services or config files from here;
    • /mnt/clone/: is simply destination of rpi-clones rsync command;
    • /lost+found/: again, seems unlikely;

    As for the remaining, I don't think there is anything in there that would prevent OMV webserver from starting when I boot up from the clone.

    • tmp
    • sys
    • proc

    That just leaves:

    • run: where I see omw-engined.pid and samba

    Nonetheless, this thread seems to indicate that it is common for the above folders to be excluded... So maybe it's none of the above.

    Also, based on OP's original comment, I did a diff command between my working install's var folder and that of the clone and found that after running:

    Code
    sudo diff /var/ /mnt/clone/var -r -q

    I got the following potential points of difference. Some missing .png's are surely not going to break things, I don't think the socket thing is the problem, but could the missing openmediavault log file really be it?

    Code
    Only in /var/lib/openmediavault/rrd: cpu-0-day.png
    ... (other missing .png files)
    Only in /var/lib/openmediavault/rrd: uptime-year.png
    ...
    File /var/lib/samba/private/msg.sock/14280 is a socket while file /mnt/clone/var/lib/samba/private/msg.sock/14280 is a socket
    ... (other similar samba socket messages)
    File /var/lib/samba/private/msg.sock/902 is a socket while file /mnt/clone/var/lib/samba/private/msg.sock/902 is a socket
    ...
    Only in /var/log: openmediavault
    ...

    Anyone know where the rsync command is in the backup plug-in's source code so I can see what I am missing with rpi-clone's rsync command?

  • Maybe https://github.com/openmediava…loy/usbbackup/default.sls contains what you are looking for?

    there are lines referencing rsync

    Code
    # Create the rsync scripts regardless if enabled or disabled. This is
    # necessary to be able to run the job manually from within the UI.
    configure_usbbackup_rsync_script_{{ job.uuid }}:

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

    Einmal editiert, zuletzt von mi-hol ()

  • G'day everyone!

    I finally had another chance to look at this again. I started reading the source code that mi-hol referenced, which led me here and here. I saw some more mentions of /var/lib and decided to just try re-resyncing /var and checking if that helped at all, before I dug any deeper into the source code. I am not sure if it was dumb luck, or maybe I had done an update since, but it worked! :D


    To be clear, I was running off of my USB drive, when I ran the regular rpi-clone command that most users would use, but without the -l ('leave') flag, to ensure that the next reboot was to the rpi's SD card:

    sudo rpi-clone mmcblk0


    TL;DR:

    However, when rpi-clone has performed its sync and I was asked:

    Hit Enter when ready to unmount the /dev/mmcblk0 partitions ...

    Before hitting enter, I opened another terminal session and ran the following:

    sudo rsync /var/ /mnt/clone/var -r

    I then returned to the rpi-clone terminal and hit Enter to unmount (the SD card) before running a sudo reboot. The rpi then booted to the SD card as expected, after which I was still able to ssh in and I was able to log in to the OMW web server via my browser :)


    BONUS:

    For those a little less confident with rpi-clone: I easily switched back to booting from the USB again without cloning the SD-->USB (just in case). This is done by running:

    1. lsblk to see the device name of your USB's root file system partition (usually the second on the drive, e.g. sda2), then
    2. blkid to get the PARTUUID of that partition (e.g. 'e458e4c8-02')
    3. sudo nano /boot/cmdline.txt and replacing the PARTUUID of your SD card with the USB's, e.g. my cmdline.txt looks like this:
      usb-storage.quirks=0781:5583 console=serial0,115200 console=tty1 root=PARTUUID=e458e4c8-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
    4. sudo reboot

    After rebooting, you can then double check that you are booted from the intended drive with another lsblk.


    Obviously the additional rsync command is a bit of a hack of a solution, I am still not sure what /var wasn't already being copied.

    If anyone knows of an additional flag that I can add to the rpi-clone command that would force the /var folder to be synced correctly, I would be eternally grateful.


    HACK JOB/WORK AROUND:

    Just to be sure, I may still use OMV's in-built backup feature to make periodic backup .iso files on my main drive. However, because rpi-clone is (normally) so convinient to switch roll back to your last working OS, I automated the above workaround by:

    • creating a new script 'backup-os' via cd /usr/local/sbin, then sudo nano backup-os
    • copying the following code into the script file
    • converting backup-os to an executable script via sudo chmod 755 backup-os
    • then modifying my existing cron job that runs the backup-os twice a week, via sudo crontab -e
    Code
    #run a backup of the OS at 2:30AM every Monday and Thursday morning:
    30 2 * * 1,4 /usr/local/sbin/backup-os

    This is hardly my area of expertise, so if anyone cares to add some constructive feedback to the above, please feel free!


    Lastly, cheers to the OMV team and all of the people contributing here. It is a thankless job, but I assure you, there are thousands of people benefiting from your efforts :)

  • Thanks DataHoarder !

    This was EXTREMLY helpful.... I started to go round myself due to this...

    It was the first time I encoutered this issue with rpi-clone not properly cloning an OS to an SD card.

    🙏🙏🙏

    ! Started hacking with ZX81 and I am getting younger everyday like Master Yoda ! ! Raspi is to young generation what ZX81 was to mine ! ! I just love Raspi! Triple bang for the Raspi Foundation ! !And thank you guys for OMV !

  • Zitat

    TL;DR:

    However, when rpi-clone has performed its sync and I was asked:

    Hit Enter when ready to unmount the /dev/mmcblk0 partitions ...

    Before hitting enter, I opened another terminal session and ran the following:

    sudo rsync /var/ /mnt/clone/var -r



    Hey datahoarder , I created an account just to thank you, you solved my problem !!


    I am super grateful.


    Let me describe --verbose my issue so maybe it helps other people find your solution on Google.


    I was trying to migrate my Raspberry Pi OMV from SD Card to SSD, using RPI-Clone.

    Everything was fine, except I was getting no web UI for OpenMediaVault.

    The reason seemed to be something related to nginx missing some data

    /var/log/nginx/access.log" failed (2: No such file  etc... Got a few like this one.


    After some omv-firstaid and omv-salt magic, I was able to fix some of the errors I was getting, and the webUI came back.

    Except I was getting this error when applying changes :


    500 - Internal Server Error Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; omv-salt deploy run --no-color postfix 2>&1' with exit code '1':


    It turns out copying /var/ before unmounting /mnt/clone/ fixed it.

    Thank you so much for thinking about this

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!