Setting up Rsync

    • OMV 4.x
    • Setting up Rsync

      I am trying to set up a scheduled Rsync job using the OMV Users Guide (page 61 & following). Here is the job I created:

      When I run the job I get the following error:

      Going back to my file systems I discovered that both of my disks are not referenced:

      How can that be when I have five shared folders attached to disk1?

      What am I doing wrong?
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • I'm not sure what was going on, but I deleted all of my shared folders and such so that I could reformat the hard drive, then recreated the users, folders, and scheduled jobs — then it all went right. Obviously, I haven't committed any real data to the beast yet so deleting and recreating wasn't a big deal. Still curious what caused it to act up like that.
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • It couldn't find the rsync command. When specifying a command, you should use the full path - /usr/bin/rsync
      omv 4.1.15 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • I was just following the example given on page 62 of the OMV User Guide. As I said above after reformatting my data drive & re-setting up my shared folders, it worked fine. But in the future I will use the full path. Thanks for the tip.
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • I guess I spoke too soon. My disk refused to stay mounted/referenced. I stumbled across this thread that explains that there is something amiss with the odroid-hc2 sata firmware that is incompatible with enabling anything on physical disk properties. I haven't had time to test this out, but will soon.

      I also came across this post on Rsync two OMV Machines that did mention "Include the whole disk in this share (path: "/")". So, @ryecoaaron, are you saying the command should look like this:

      /usr/bin/rsync -av --delete /srv/dev-disk-by-label-disk1/ /srv/dev-disk-by-label-disk2
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • Since it specifies the full path to the command, the following would work, if you chose to use it.
      /usr/bin/rsync -av --delete /srv/dev-disk-by-label-disk1/ /srv/dev-disk-by-label-disk2

      However:
      Linux has the equivalent of Windows paths to executables, so app's in user binaries(/usr/bin/) should run without specifying the full path. In essence, "rsync" , as a standalone command, should work.

      If rsync exists under /usr/bin and the command doesn't work, as it is in the guide, you might have something else going that needs attention.

      I'd try to figure out why your disk(s) won't stay mounted/referenced. If disks aren't mounted, when the command executes, rsync can't work (with or without the full path).
      _______________________________________________________________________________

      While it's possible to share the root of a drive "/", I'm not a fan of doing it.

      If you're not the only user of your NAS:
      Sharing the root of a data drive can set up unintended permissions issues where, among potential outcomes, LAN users may be able to delete the entire contents of a data drive.

      Some do it anyway but, without tailoring permissions, sharing the root adds risk. (That's just my opinion - your call.)

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • I set it up with the /usr/bin/rsync path and it seems to be working fine. Give me a clue: how do I tell if I'm getting a good copy on the backup drive? Total novice on this end; probably should take a class in Linux command line.

      I am pretty sure the unmounting is related to customizing spindown and such. This time around I just left the options at disable when setting up the drives. That is what I read in a post (I linked to it above).

      I don't intend to share with this server. Just me. All the machines on the LAN are mine. But I'm open to suggestions. Never know what may change in the future.

      Thanks for the info.
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • When doing a disk to disk backup, the first copy might take awhile. Depending on the amount of data, the first sync may take hours to even days. During the first job, misc system events might interrupt the job (a reboot for example) but, with the switches set on the command line, Rsync will retain the files copied and pick up where it stopped on the next scheduled event.

      If you're wondering if you have a full disk-to-disk sync, run the job manually. (Note, if you have a massive directory and file structure, it may take a couple minutes for Rsync to gather info and start.)
      If Rsync starts and begins to process files, fine, let it run. (And if you recycle the browser window, the rsync job will continue in the background while you do something else.)
      If the job ends without file processing, and you see just a few bytes sent and received, that's directory information. The drives are in sync. At that time, from a file and folder perspective, both drives are functionally identical.

      When your drives are in sync, future jobs will consist of added, deleted, or changed files only - very fast and efficient.
      __________________________________________________________________

      Unless you see something in the log (Diagnostics, System Logs, Rsync-Jobs) that indicates there was a problem, you can trust the backup. Rsync has been around a long time. It's well tested and very reliable.
      __________________________________________________________________

      Even if you are the sole user of your server, it's still not a good idea to share the root "/" of a data drive. (Again, my opinion.) Anything done at the drive level comes with a risk of human error, "fat finger" mistakes, that can result in the loss of the entire data store. That's the reason for the stern warnings, I called them "sanity checks", on the drive-to-drive backup routine in the guide. Where possible, drive level operations should be avoided or done with extreme care.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • I had my Rsync set to backup my Odroid-hc1 in the wee hours of Monday. Today was my first go. My Cron Daemon notifications woke me early this morning with all kinds of bad news. One message started like this:

      mesg: ttyname failed: Inappropriate ioctl for device
      rsync: readlink_stat("/srv/dev-disk-by-label-disk2/AppData/Plex/Library/Application Support/Plex Media Server/Media/localhost/0/cdf00387e8d9c0d835bbaf701618a8d2c6004f7.bundle/Contents/Thumbnails/thumb1.jpg") failed: Bad message (74)
      rsync: recv_generator: failed to stat "/srv/dev-disk-by-label-disk2/AppData/Plex/Library/Application Support/Plex Media Server/Media/localhost/0/cdf00387e8d9c0d835bbaf701618a8d2c6004f7.bundle/Contents/Thumbnails/thumb1.jpg": Bad message (74) ...

      plus much more. A hundred or so of these inter laced with notifications that start like this:

      sending incremental file list
      AppData/Plex/Library/Application Support/Plex Media Server/Cache/
      AppData/Plex/Library/Application Support/Plex Media Server/Cache/CloudAccess.dat
      AppData/Plex/Library/Application Support/Plex Media Server/Logs/
      AppData/Plex/Library/Application Support/Plex Media Server/Logs/Plex Media Server.log
      IO error encountered -- skipping file deletion
      AppData/Plex/Library/Application Support/Plex Media Server/Media/localhost/3/0149f228a82afa4b6fd8f23457630b578a39cf7.bundle/Contents/...

      And on it goes. There were a number of these mixed in too:

      Execution failed Service nginx

      Date: Mon, 19 Nov 2018 06:03:44
      Action: alert
      Host: omv1
      Description: failed to stop (exit status -1) -- Program '/bin/systemctl stop nginx' timed out after 30 s

      And then several like this:

      Resource limit matched Service omv1

      Date: Mon, 19 Nov 2018 05:51:48
      Action: alert
      Host: omv1
      Description: cpu system usage of 97.4% matches resource limit [cpu system usage>95.0%]

      There were about 300 emails in all. I am unable to load the web GUI of either omv or Plex. Trying to ssh in times out. Do I just unplug it and throw away the micro SSD and start from scratch?
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.

      The post was edited 1 time, last by Agricola ().

    • On the rsync errors:
      You know that Plex may be working, potentially around the clock, collecting metadata for your media files. Along those lings, the first thing rsync does when a job is started is to scan the source and destination, to generate a files list to copy.

      With that in mind, what do you think would happen if the contents of the source disk is changed, after rsync scans it? (Files added, changed or deleted.) Maybe a few errors? Until things settle out with your install and Plex has a chance to finish collecting data for your media files, I'd ignore errors related to Plex.

      Under System, Notification, Notifications ; I'd uncheck everything except filesystems and SMART.
      On weak hardware, you will run the CPU near 100%, stretch memory, etc., etc., and it will happen on a fairly regular basis. On the other hand, Filesystems and SMART is useful in that it might alert you before a drive dies completely.

      If you get E-mail with; ttyname failed: Inappropriate ioctl for device - search the forum for the fix.

      Don't know what to tell you about Execution failed Service nginx
      If you can get into the web console and other OMV hosted web pages, I wouldn't worry to much about it.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • I wouldn't bother backing up the PLEX metadata folders. Instead make sure that the video files are backed up. In case all of the plexmediaserver files are lost plex can be reinstalled easily and automatically update and download the metadata again. But most likely the actual media files are not as easy to replace.

      If you want FULL backups then you most likely have to stop everything running on the nas. The easiest way is to boot from some other media and mount the drives and backup that way.

      Also make sure that you have versioned backups. Timestamped rsync snapshots. If you have a single copy of all the files, and update that copy daily, then errors and problems are propagated to the backup copy. Almost as fast as if you used RAID.

      I don't use the OMV GUI for any backups. At all. I assume that it works fine, but I don't know exactly how and I don't feel in control. I use simple scripts that I have written and debugged and tested over the years and that I understand in detail. The scripts use rsync to create versioned rsync snapshots. And I run the scripts using cron jobs. I've done backups of Linux computers like this for a long time, since long before I knew OMV even existed.

      Also I organize my filesystems so that it is easy to backup just the stuff I want to backup. And not stuff that doesn't need to be backed up. It saves a lot of space on the backup media and makes it possible to keep MANY versions of the important parts of the filesystem that is backed up.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh
    • At this point I am still not able to do anything with the hc1. Ssh returns port 22: Operation timed out, and the web gui isn't there for Plex or omv for two days now. Right now I just want to shut it down gracefully and start over with a clean install, but don't know how to. I guess I'll just pull the plug unless someone has a magic wand.

      @Adoby thank you for the info. I will look at your backup scheme as soon as I get back up and running. You will have to explain a bit further the scripting and the filesystems organization. I am sorry but I am not familiar with it.

      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • If you're starting over, there's no need to worry about a clean shutdown. Just pull the plug. You might want to reboot it, if it will come up, just to see what happened. (Check the logs.)

      If you start over:
      In the guide, there's guidance about how to clone your boot media. The second time around, you might want to get two boot devices (SD-cards) and after you get a clean build, clone it. Then have at configuring it up. If something goes wrong, with a clone, you won't have to start from scratch.

      If all goes well and you have a good, fully configured NAS with all working as it should, the second SD-card will let you back up the final product.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • I'm sorry. I should have said that I made a disk image using "dd" about a week ago. It just bothers me to just pull the plug. I know an SD card degrades that way. I had hoped that it would respond at some time, but I figure I have waited long enough. Thanks.
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.
    • Regarding rsync backup scripts.

      I recommend that you write your own. Test it and make sure you understand what it does. And what it doesn't do. Then you can use that script again and again with confidence.

      If you Google "rsync backup script" you get many hits. You can most likely find a good starting point that way. There are many wierd and wonderful backup scripts out there.

      If you follow references and sources backwards you tend to end up at this very old site:

      mikerubel.org/computers/rsync_snapshots/

      There you can read about how early versions of rsync were improved to allow you to do rsync snapshots in a single command. That site was my starting point.

      I have recently (around 6 months ago) added improved rotation of the old snapshots to my script. I used to simply keep 30 daily snapshots or purge manually now and then. Now it is based on promotions and successive winnowing as the snapshots age. I keep all snapshots for a week. Then one snapshot per week for a month. Then one snapshot per month for a year. Not perfect yet, still testing. But it is already in use daily.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh
    • @Adoby: I appreciate your help with this. Unfortunately, your recommendations are quite a bit beyond my skill level at this time. Your backup system, especially the versioning aspects, are very appealing to me. I won't let it go until I have figured it out. Right now, though, I need something set up to protect data I am about to commit to an hc1.

      @flmaxey: I'm back up and running, but my os backup wasn't as far along as I had hoped. Still it wasn't a total rebuild. Os is backing up as I write. When backed up, I plan to try the rsync schedule again as lined out in the User Guide. I gather that you have a hand in the creation/editing of the User Guide. In future editions would it be possible to throw a bone now and then to users laboring under Mac machines. When I backup my SD cards I use Terminal. It took me some digging to figure it out: sudo dd if=/dev/disk# of=~/backupSD/hc1omvyyyymmdd.dmg, where disk # is discovered using diskutil list.


      Still, it is a marvelous Guide for beginners like me. I'm looking forward to the addition of backing up to a remote machine. If I understand the recovery part of rsync, wouldn't this be an acceptable way to move to a larger drive: rsync to a larger drive on usb and then redirect the shares to it, then unmount drives, shutdown, switch backup to sata and then reboot?
      Retired. Love to garden and mess with computers. The more I mess with both the less I know about either.
      OMV 4.1.14-1, Odroid hc2 w/ 4TB WD Blue.

      The post was edited 1 time, last by Agricola ().

    • Agricola wrote:

      @flmaxey: I'm back up and running, but my os backup wasn't as far along as I had hoped. Still it wasn't a total rebuild. Os is backing up as I write.
      Good! As you work through things and take wrong terms, your OS backup will save a lot of time. Hint: If you name the image file, for example: OMV-Od-23NOV18.IMG that's yet another OS backup. I run an MD5 hash on the image file and store the hash and the image in a 7ZIP archive. (This greatly reduces the needed storage space.)
      Later on when all is working well, if an update goes sour or something else goes wrong, OS backup with save a ton of rebuild/reconfig time.

      Agricola wrote:

      I gather that you have a hand in the creation/editing of the User Guide. In future editions would it be possible to throw a bone now and then to users laboring under Mac machines.
      Yeah, I wrote it with info collected from the forum, and my own experiences along the way. It seemed clear that new users needed something with more detail to get them started. But it's not perfect, and a work in progress as things change.

      I would throw Mac Owners a bone but,, the guide is a walk-through type reference and I don't have a Mac. Maybe if I can find a cheap Mac-mini on Ebay, something that's reasonably up to date with a version of OSX. I don't know, we'll have to see. Maybe there's a Mac VM possibility? (I'd be in a learning curve with a Mac. :) )

      Agricola wrote:

      I'm looking forward to the addition of backing up to a remote machine.
      That's coming soon, when we're snowed in or when it's icebox cold. (Inevitable) A second server on the local network, can be configured to take over as a replacement server with the same content as the primary. Other than having something off site, this is the ultimate backup - a full data copy on an independent platform. The primary could crash and burn, and you'd still be OK.

      Agricola wrote:

      If I understand the recovery part of rsync, wouldn't this be an acceptable way to move to a larger drive: rsync to a larger drive on usb and then redirect the shares to it, then unmount drives, shutdown, switch backup to sata and then reboot?
      You're getting the idea. Yes, it should work. My only concern would be how the Odroid names devices. But since you're using OMV's "drive name by label" in the rsync command line, the device name (/dev/sdb1 for example) shouldn't matter.

      Drive to drive rsync can also be used to transfer data between drives with different format types (EXT4 to BTRFS) and moving data to larger drives, while maintaining permissions, attributes, etc. There's great utility in it.

      All I can say is be CAUTIOUS about which drives are in the source and destination. Setting an empty drive in the source, will erase the destination. Check the command line, recheck it, then check it again before using it.
      ______________________________________________________________

      @Adoby is right, however, about looking at more sophisticated back up. As your level of experience grows you should look as something like the RSNAPSHOT plugin. I'd like to write an addendum for RSNAPSHOT, and a couple other utility plugins, time permitting.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 1 time, last by flmaxey: edit ().