Rsync via GUI on OMV

    • OMV 4.x
    • Resolved

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Rsync via GUI on OMV

      I have OMV 4.x installed on a Raspberry PI with two USB connected SSDs. USB devices are self powered.

      I have it configured and SMB access on one drive with the data. All works great. My intent is to Rsync every night from SSD1 to SSD2. Essentially a mirror copy of the data.

      Is there a guide on how to create Rsync via the GUI? I mean what do those settings for the time, day, date etc mean? Default is *. What does * meaN? I created a job but it would not trigger. If I manually attempt to run it ran (I think it did, but I cancelled since I wanted to test the automation). My intent is to trigger the rsync operation every night at 2 AM.

      I am somewhat familiar with Rsync on the command line, But I am setting this up for a friend of mine. His use case is simple. He wants some network drive on his wifi to dump all the pics & videos he clicks with his phone. And he wants two copies.

      Any ideas on how to set up Rsync to trigger an rsync operation at a particular time of day?
    • The standard Linux way of doing this is:

      Create a script, with the rsync command in it, like on the command line.
      Add a line in the file /etc/crontab that run the script at certain times.

      The GUI in OMV does the same, but instead of using any paths as source and destination for rsync, you use shared folders. This is unique for OMV.

      And you specify when to run the script using similar elements as in /etc/crontab. But in a GUI.

      And you set rsync options in the GUI as well.

      * means every. So you typically specify a time and then every everything else. Then rsync run every day at that time. Or every Monday at that time. Or whatever.

      I think the rsync GUI is silly and clumsy. I never run rsync with whole shared filesystems as source or destination. I run rsync with some specific folders as source and destination. And I think it is much simpler to add a line in /etc/crontab than starting the OMV GUI to do it. But I suppose some are allergic to the command line...

      And I use rsync to create versioned snapshots, with hardlink. And my script also purge old snapshots.

      There are plenty of online tutorials on how to use crontab to run scripts at certain times. Just search for "crontab tutorial" or something.

      And here is the rsync script I use: github.com/WikiBox/snapshot.sh

      This script should be ideal for your purposes. It can create several timestamped backup copies. Each copy looks like a full copy, but only takes up enough space to store changes from the previous copy/snapshot.
      OMV 4: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

      The post was edited 1 time, last by Adoby ().

    • vishnumrao wrote:

      I have OMV 4.x installed on a Raspberry PI with two USB connected SSDs. USB devices are self powered.

      I have it configured and SMB access on one drive with the data. All works great. My intent is to Rsync every night from SSD1 to SSD2. Essentially a mirror copy of the data.

      Is there a guide on how to create Rsync via the GUI? I mean what do those settings for the time, day, date etc mean? Default is *. What does * meaN? I created a job but it would not trigger. If I manually attempt to run it ran (I think it did, but I cancelled since I wanted to test the automation). My intent is to trigger the rsync operation every night at 2 AM.

      I am somewhat familiar with Rsync on the command line, But I am setting this up for a friend of mine. His use case is simple. He wants some network drive on his wifi to dump all the pics & videos he clicks with his phone. And he wants two copies.

      Any ideas on how to set up Rsync to trigger an rsync operation at a particular time of day?
      Scheduling Rsync jobs is about the weirdest thing ever (IMO).... I'll use mine to try and provide some clarity (as it is pretty similar to what you want to do)

      The N trigger, means its going to do that job every N. So if you set hours to 8, and enable the N trigger, that means it will run every 8 hours. If the N trigger is not set, it will run every day at 8am (as I have below). The confusing part is, if you set the hours.. it's still effected by the minutes. Example... You set the hours for 8, enable the N trigger, then set the minutes to 15... That means every day at 8am, that job is going to run over and over for 15min. By the same token, if you enable the N trigger both on hours and minutes, the job will run every day at 8:15am If you leave the star on Minutes.. It's going to run over and over starting at 8am until the 8am hour ends (60 minutes). When you see people posting full OS disks and it's found to be due to rsync, this is why...They've had a job running for weeks and their rsync logs have filled the OS disk.

      So... let's say I wanted to run a job every Friday at 8am... I'd set the Day to Friday, time to 8, and the minutes to 0. If I leave the minutes at * or put a value there... it's just going to run over and over at 8am on Friday, for 60min or for the value I put there... everything else would be set to * so it's ignored

      Looking at my job below... It runs every day at 0800. The minutes being set to "0".. means it runs at the top of the hour, and then doesn't run again until 0800 the following day. If you want it to run say, every 6hrs... You'd set hours to 6, enable the N trigger on hours, then set the minutes at 0. That would mean it runs every 6hrs at the top of the hour.

      Usually after I set up my jobs (as I had issues figuring this out several years ago)... I watch my rsync jobs log pretty closely for a few days to make sure it's doing what I want.

      A look at my rsync log from the last couple days, confirms the job is working properly...

      Source Code

      1. 2019/09/30 08:00:01 [28377] building file list
      2. 2019/09/30 08:00:11 [28377] sent 1,397,480 bytes received 6,050 bytes 133,669.52 bytes/sec
      3. 2019/09/30 08:00:11 [28377] total size is 1,321,544,555,456 speedup is 941,586.25
      4. 2019/10/01 08:00:01 [8149] building file list
      5. 2019/10/01 08:00:10 [8149] .d..t...... Torrents/transmission-home/
      6. 2019/10/01 08:00:10 [8149] >f.st...... Torrents/transmission-home/dht.dat
      7. 2019/10/01 08:00:10 [8149] >f.st...... Torrents/transmission-home/settings.json
      8. 2019/10/01 08:00:11 [8149] >f.st...... Torrents/transmission-home/transmission.log
      9. 2019/10/01 08:00:11 [8149] sent 3,062,173 bytes received 6,121 bytes 292,218.48 bytes/sec
      10. 2019/10/01 08:00:11 [8149] total size is 1,321,544,556,447 speedup is 430,709.89
      11. 2019/10/02 08:00:01 [8013] building file list
      12. 2019/10/02 08:00:15 [8013] .d..t...... Torrents/transmission-home/
      13. 2019/10/02 08:00:15 [8013] >f.st...... Torrents/transmission-home/dht.dat
      14. 2019/10/02 08:00:15 [8013] >f.st...... Torrents/transmission-home/settings.json
      15. 2019/10/02 08:00:15 [8013] >f.st...... Torrents/transmission-home/transmission.log
      16. 2019/10/02 08:00:15 [8013] sent 3,063,548 bytes received 6,118 bytes 211,701.10 bytes/sec
      17. 2019/10/02 08:00:15 [8013] total size is 1,321,544,557,825 speedup is 430,517.38
      18. 2019/10/03 08:00:02 [3101] building file list
      19. 2019/10/03 08:00:10 [3101] .d..t...... Data/config-backup/
      20. 2019/10/03 08:00:10 [3101] .d..t...... Torrents/transmission-home/
      21. 2019/10/03 08:00:10 [3101] >f..t...... Torrents/transmission-home/settings.json
      22. 2019/10/03 08:00:11 [3101] >f.st...... Torrents/transmission-home/transmission.log
      23. 2019/10/03 08:00:11 [3101] sent 3,064,022 bytes received 6,104 bytes 323,171.16 bytes/sec
      24. 2019/10/03 08:00:11 [3101] total size is 1,321,544,559,034 speedup is 430,452.87
      Display All
      The only other major thing to check... is the delete trigger. I personally leave it off... So if I accidentally delete something on "Disk_1", it's still on "Disk_2", and it's just simple to drop to command line and manually copy the file back (I don't use Disk_2 on any services other than rsync). Then every couple weeks, I log into the webUI, enable the delete trigger and manually run the job to bring the drives 100% in sync. Once that's done, I turn the delete trigger off again.

      Hope that helps.
      Images
      • rsync.png

        42.97 kB, 964×616, viewed 82 times
      Air Conditioners are a lot like PC's... They work great until you open Windows.

      The post was edited 1 time, last by KM0201 ().

    • i tried the instructions from KM0201. But no luck. nothing shows in the rsync jobs logs.

      i see the entry in the /etc/cron.d/openmediavault-rsynd

      Source Code

      1. root@raspberrypi:/etc/cron.d# cat openmediavault-rsync
      2. SHELL=/bin/sh
      3. PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
      4. # m h dom mon dow user command
      5. 0 20 * * * root /var/lib/openmediavault/cron.d/rsync-745097b0-1360-414c-9261-78c fab989fab >/dev/null 2>&1
      6. root@raspberrypi:/etc/cron.d#


      as an experiment set up job for 8 pm. any ideas?
    • It looks OK.

      If You've got the device now... I'd change the minutes to "15", and set the N trigger, and set hours to *... and save the job.

      Then go to System Logs/Rsync Jobs and clear the log. Wait about 30min, and check the log, and see if the job ran a couple of times like it was supposed to.
      Air Conditioners are a lot like PC's... They work great until you open Windows.

    • I set up the following:
      N trigger
      minutes to 5
      trial run only (my target directory has a years worth of pics, also backed up on a separate hdd).

      I am able to get it to trigger. I am seeing entries in the rsync jobs log 5 minutes apart.

      So somethings wrong when only time is set without the N trigger.

      Edit: I removed the N trigger and rsync does not trigger.

      The post was edited 1 time, last by vishnumrao ().

    • vishnumrao wrote:

      I set up the following:
      N trigger
      minutes to 5
      trial run only (my target directory has a years worth of pics, also backed up on a separate hdd).

      I am able to get it to trigger. I am seeing entries in the rsync jobs log 5 minutes apart.

      So somethings wrong when only time is set without the N trigger.

      Edit: I removed the N trigger and rsync does not trigger.
      in that case (N trigger off)... Assuming hours is set to *, at the top of the hour it will run over and over for 5min
      Air Conditioners are a lot like PC's... They work great until you open Windows.