Setting up Incremental remote backups to OMV?

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Setting up Incremental remote backups to OMV?

      A long time Linux user as blue in NAS as possible. I started setting up OMV 2.2 2 weeks ago to replace a desktop "server", using a tiny Intel x86 netbook. Have Syncthing's working (plugin). Next comes incremental backup of the other machines all running GNU/Linux OSes.

      I do it from the desktop via rsnapshot & ssh atm. Pushes backups of the remote hosts daily, weekly and monthly to HDD "Uno" and "Deuzio" that are swaped every week. Swapped HDD being stored offsite.
      The few non-Linux OSes (Android and a Windows laptop) is sync'ed with Syncthing for our data; and with rsync or imaging for the systems confs.

      I'd like to hear those of you guys who do incremental data backup from remote hosts say a bit how do you manage that via OMV!?
      Before I start to set up rsnapshot to do auto incremental backups of the remote host the usual Linux way (setting up password-less key auth). As I may well be missing the NAS way, new to me as it is.
    • Okay, would like to hear how you guys push incremental backups from OMV. There might be a couple of plugins I overviewed.

      I added rsnapshot-once to rsnapshot. Now enjoying automatic incremental backup of all my personal non audio-video files as well as users and systems configs (~20 gig) from a desktop, a laptop and OMV itself. It's set up so that any new/changed content on any machines will be backed up at least once a day, provided the box's ON a couple of hours.
      As for restoration I tried NFS share. Wasn't smooth so I've set up sshfs on the external boxes and it works fine.
    • Here is what I do:

      Th goal is to back up a remote server onto omv and keep daily snapshots of the backups

      Prepare key on OMV:
      1. Open Web Gui and go to System -> Certificates and create a new ssh Certificate. The result should look like this:
        [IMG:http://forums.openmediavault.org/index.php/Attachment/4187-NewCert-PNG/]


      Prepare remote-server
      1. Create a user remote-backup on the server
      2. allow user remote-backup to execute rsync as root:

        Source Code

        1. cat <<EOF > /etc/sudoers.d/remote-backup
        2. # remote backup
        3. # allow user remote-backup to execue rsync as root
        4. remote-backup ALL= NOPASSWD:/usr/bin/rsync
        5. EOF

      3. create a file /home/remote-backup/bin/validate-backup-cmd.sh

        Source Code

        1. su remote-backup
        2. cd ~
        3. mkdir bin
        4. cat <<EOF >bin/validate-backup-cmd.sh
        5. #!/bin/bash
        6. # echo "${SSH_ORIGINAL_COMMAND}" >> ~/remote-backup.log
        7. case "$SSH_ORIGINAL_COMMAND" in
        8. *\&*|*\|*|*\;*|*\>*|*\<*|*\!*)
        9. echo invalid command
        10. exit 1
        11. ;;
        12. /usr/bin/rsync\ --server\ --sender*)
        13. sudo $SSH_ORIGINAL_COMMAND
        14. ;;
        15. rsync\ --server\ --sender*)
        16. sudo /usr/bin/$SSH_ORIGINAL_COMMAND
        17. ;;
        18. *)
        19. echo invalid command
        20. exit 1
        21. ;;
        22. esac
        23. EOF
        24. chmod +x ~/bin/validate-backup.cmd.sh
        Display All

        This checks the remote command and only allows commands starting with rsync --server --sender to be executed

      4. Allow execution of the new script via ssh:

        Source Code

        1. mkdir ~/.ssh
        2. chmod 700 ~/.ssh
        3. nano ~/.ssh/authorized_keys
        and
        edit this file (inserting the pblic key you previously created)

        Source Code

        1. command="/home/remote-backup/bin/validate-backup-cmd.sh" ssh-rsa <some public key>

        This will only allow remote-backup to execute the shell script /home/remote-backup/bin/validate-backup-cmd.sh and nothing else
      5. Make sure the file ~/.ssh/authorized_keys is only readable by its owner, otherwise you will get indeciperable errors later:

        Source Code

        1. chmod 600 ~/.ssh/authorized_keys

      On OMV:
      1. Go to Services -> rsync and create a new rsync Job
        [IMG:http://forums.openmediavault.org/index.php/Attachment/4188-RsyncJob-PNG/]

        Extra options is: --numeric-ids -q -e '/usr/bin/ssh -oStrictHostKeyChecking=no -p 22 -i /var/lib/openmediavault/ssh/keys/openmediavault-1c69f786-25aa-4941-b5b6-4ef49514ec26' --delete-excluded --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/run/* --exclude=/lost+found --hard-links

        consult a rsync man page for the rsync options. The -e '...' tells rsync to use ssh as transport not rsync. The options given to ssh are
        1. -oStrictHostKeyChecking=no Accept the remote servers key without prompting
        2. -p 22 Use port 22 (default)
        3. [-i /var/lib/openmediavault/ssh/keys/openmediavault-1c69f786-25aa-4941-b5b6-4ef49514ec26] use the key created before
          (might be a bit tricky to find it, but it is in the given directory)
      2. Add a rsnapshot job (Services -> Rsnapshot -> Add) to do daily incremental backups of the share above to some other share (e.g. snapshots)
        [IMG:http://forums.openmediavault.org/index.php/Attachment/4189-Rsnapshot-PNG/]


      I am using this to back up several remote servers, but if you have a database running on one of them you should do a local backup before you rsync it.
      Images
      • NewCert.PNG

        52.69 kB, 655×456, viewed 3,653 times
      • RsyncJob.PNG

        62.38 kB, 1,193×911, viewed 4,111 times
      • Rsnapshot.PNG

        47.31 kB, 916×621, viewed 3,998 times

      The post was edited 1 time, last by Zoki ().

    • Well, I followed through, but when I execute the job, I get this error:

      Source Code

      1. Please wait, syncing <remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold> to </media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/> ...
      2. edly closed (0 bytes received so far) [Receiver]
      3. rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]


      I did manually run rsync and it was successful:

      Source Code

      1. root@omv-pd:/etc# rsync -avz -e ssh remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold /media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/
      2. remote-backup@openfiler299's password:
      3. receiving incremental file list
      4. MusicOnHold/
      5. MusicOnHold/ohm1106001.nch
      6. sent 34 bytes received 1195197 bytes 341494.57 bytes/sec
      7. total size is 1215886 speedup is 1.02


      Now what?

      Also, I'm not sure where the /home/remote-backup/bin/validate-backup-cmd.sh command is supposed to come into play?

      How might I manually run the rsync command with the extra options to verify the command line? When I try that, it's prompting for a password, which would imply I don't have the keys set up right?

      Source Code

      1. root@omv-pd:/var/lib/openmediavault/ssh/keys# rsync -avz -e ssh --numeric-ids -q -e '/usr/bin/ssh -oStrictHostKeyChecking=no -p 22 -i /var/lib/openmediavault/ssh/keys/openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39' --delete-excluded --exclude=/lost+found --hard-links remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold /media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/
      2. remote-backup@openfiler299's password:


      sure enough, if I manually enter the password, rsync transfers files successfully....though no output. I guess I'm not understanding how the key exchange is supposed to work.

      The post was edited 3 times, last by bkeadle ().

    • Not sure about the error from the remote side, but in you last command you do set the -e flag twice.
      Remove the first -e ssh and try again (and to debug, remove the -q flag (quiet)):

      Source Code

      1. rsync -avz --numeric-ids -e '/usr/bin/ssh -oStrictHostKeyChecking=no -p 22 -i /var/lib/openmediavault/ssh/keys/openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39' --delete-excluded --exclude=/lost+found --hard-links remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold /media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/
    • I got that command from as revealed from OMV's cron job.I set up key exchange for root user, and changed the job to login as root and removed the extra options and it works as expected.

      I'm sure now that the problem is with trying to ssh as my remote-backup user...and I'm prompted for a password.

      I even tried this:

      Source Code

      1. root@omv-pd:/var/lib/openmediavault/ssh/keys# ssh-copy-id -i openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39.pub remote-backup@openfiler299
      2. remote-backup@openfiler299's password:
      3. Now try logging into the machine, with "ssh 'remote-backup@openfiler299'", and check in:
      4. ~/.ssh/authorized_keys
      5. to make sure we haven't added extra keys that you weren't expecting.
      6. root@omv-pd:/var/lib/openmediavault/ssh/keys# ssh remote-backup@openfiler299
      7. remote-backup@openfiler299's password:


      and still getting prompted for remote-backup's password

      The post was edited 1 time, last by bkeadle ().

    • I did not exactly what you did.

      can you execute

      Source Code

      1. ssh -i /var/lib/openmediavault/ssh/keys/openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39 remote-backup@openfiler299

      -i is the path to the _private_ key file of the user trying to log in.

      The result should be an error message about your command not beeing allowed.
    • Hi,


      great !

      it sounds that it"s exactly what i'm searching;

      But need more explanations.

      datruche, you mentioned using rsnapshot. my goal is to backup my raspberry on my nas. So it could be possible with that ?

      Zoki, i tried your solutin, it works except the line


      Source Code

      1. command="/home/remote-backup/bin/validate-backup-cmd.sh" ssh-rsa <some public key>
      Sorry i d'ont know how use it ... Do i have to replace all the "ssh-rsa <some public key>" by my key ?


      Finally i wonder what is the best solution ?

      using rsync or rsnapshot with OMV?

      Thanks for your help,
      JD
    • (Note: a 3.X version of OMV is required to use the UrBackup plug-in.)
      _____________________________________________________________________

      I've been using Rsync to backup the data directories of a file server. The way I'm doing it uses the "Remote Mount" plugin, so the remote shares appear to be local to OVM.



      After sharing the mount points created above, Rsync is set up below.


      Under source, "Backups_R" means "remote". The destination is a share on the data drive of OMV. Videos_THPC is another client where a good number of large occasionally changing files reside.
      _______________________________________________________________________________

      I've been using UrBackup for a couple clients, for both full and incremental image and file backups. It works fine. However, since I'm using a Raspberry PI for both operations, for file server and client backup, I limit concurrent "UrBackup" backups to one as a time.



      With a 100mbs interface and an Arm processor, avoiding parallel operations whenever possible just makes sense. On the other hand, it's a very low cost, low power solution (about 12 watts) where I have cloned OMV boot media backups and hardware replacements in hand.

      With a few minor bumps to navigate, a NOOB (like myself) can do this.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)

      The post was edited 2 times, last by flmaxey ().

    • LarsN wrote:

      David B wrote:

      I know this thread is old, but if anyone is looking for a different way to go about doing this then Duplicati is an option as well.
      People are working on an Duplicati plugin for OMV. It's still in beta.omv-extras.org/joomla/index.php/omv-plugins-3/3-testing (scroll down a bit)
      I thought I remembered seeing a Duplicati plugin for OMV in the works somewhere before, but I did not want to say anything about it because I was not sure of it.
    • The duplicati plugin isn't reall beta. It is too simple. duplicati itself kind of is.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!