Simplest way to sync files between OMV instances

    • OMV 4.x
    • Simplest way to sync files between OMV instances

      Hi,

      I'm trying to find the easiest way, to sync specific folders between separated OMV installations. The installations are on different networks. Rsync, Nextcloud, Syncthing, other services? My requirements are one-way and two-syncs and done automatically by a fixed schedule or when some file changes are detected.

      Before I start implementing anything, I'm trying to do some research...

      Thanks for some feedback!
    • pro_lamer wrote:

      Shinobi wrote:

      Hi,

      different networks.
      how much different are they? Is there internet between them? What bandwidth available and needed both directions?
      Sent from my phone
      Yes, there is internet in between. The bandwidth won't be the issue. Maybe I should also mention that I would act like a "master node" and want to share different folders with different other instances. Some might be for multiple instances, some might be only for one instance.

      I watched a bit about Synthing and so far I think it could do what I need it to and it seems to have an intuitive interface

      I will also take a look at Resilio. Thanks
    • I would use rsync.

      If you don't mind following a Youtube video, it would also be pretty easy setting up Syncthing on each OMV box. I set it up just to do it, and it was simple to set up. I've always used and been happy with rsync so I use it vs Syncthing.

      youtube.com/watch?v=paNd4hqG9Zk&t

      For ease of use, I'd probably give a very slight edge to Syncthing, as it's a little easier to set up. The main thing with rsync, is it takes some practice getting the scheduling right. Other than scheduling, rsync is a little easier to set up, if you're setting it up with the webUi.

      Rather than RAID, I run my 2 NAS disks independently. I set up an automatic rsync job to sync the two drives a couple times a day. I have 2 off site NAS boxes (they're both oDroid HC2's) that are single drive. One is at my Mom's, which is a full backup that runs once a night. The other sync job runs after my full backup, and backs up Movies and TV Shows to my sisters where I also set up a simple SMB share so she can watch them on her Android TV box.

      Been extremely happy with this setup.

      Hope that helps.
      Air Conditioners are a lot like PC's... They work great until you open Windows.

      The post was edited 1 time, last by KM0201 ().

    • For unidirectional sync, rsync is probably the best choice (as others have said)

      I need bidirectional sync for my use case, and I currently use Unison for bidirectional sync between two OMV NASes in two locations. I will soon be rebuilding these NASes, however, and will probably move away from Unison.
      • Unison advantages: bidirectional, based on rsync-type protocol, can sync file user/group, lots of include/exclude flexibility
      • Unison disadvantages: Versions must match on both servers (generally not a problem if you are running the same version of Debian on both sides), not very tolerant of network outages, requires a direct connection between servers (no assisted discovery), so dynamic DNS is required or some other means of knowing the IP address of the slave.
      I'm considering moving to Syncthing. The newest versions of Syncthing have limited ability to sync file user and group, eliminating the need to run a separate sync process for each user.
      • Syncthing advantages: Can now sync file user/group in a limited way (only the most expensive "enterprise" version of Resilio can do this), supports discovery servers, so no need for dynamic DNS, more tolerant of network outages.
      • Syncthing disadvantages: Can't handle all possible cases of syncing file user/group.
      For bidirectional sync, another possibility that I have not yet investigated is Osync, which is rsync wrapped in Bash scripts to provide bidirectional syncing and some other features.
    • KM0201 wrote:

      I would use rsync.

      If you don't mind following a Youtube video, it would also be pretty easy setting up Syncthing on each OMV box. I set it up just to do it, and it was simple to set up. I've always used and been happy with rsync so I use it vs Syncthing.

      youtube.com/watch?v=paNd4hqG9Zk&t

      For ease of use, I'd probably give a very slight edge to Syncthing, as it's a little easier to set up. The main thing with rsync, is it takes some practice getting the scheduling right. Other than scheduling, rsync is a little easier to set up, if you're setting it up with the webUi.

      Rather than RAID, I run my 2 NAS disks independently. I set up an automatic rsync job to sync the two drives a couple times a day. I have 2 off site NAS boxes (they're both oDroid HC2's) that are single drive. One is at my Mom's, which is a full backup that runs once a night. The other sync job runs after my full backup, and backs up Movies and TV Shows to my sisters where I also set up a simple SMB share so she can watch them on her Android TV box.

      Been extremely happy with this setup.

      Hope that helps.
      Yes, thanks. That is also a big part of my use case... Like I said, I'm still in the play around phase and will both give a try. Probably setting up 2-3 OMVs and sync around in my home network :D
    • macom wrote:

      David Rosky wrote:

      bidirectional sync
      Can't this be achieved with two rsync jobs using appropriate switches?
      I'm not an expert with rsync, but it seems possible to do since Osync uses rsync, but I'm not sure how to do it. You might want to check out Osync ( github.com/deajan/osync ), which claims to do bidirectional sync using rsync and Bash wrapper scripts. It looks interesting and I will probably evaluate it before deciding on what to use to replace Unison.