Best practise for backing-up/snapshotting a remote smb share to omv

Hey there,

I'm running a couple of omv setups and struggeling right now a little bit.

I used to (omv2.x) use the remote share plugin to mount a smb (and webdav) share localy in omv.
Then I used the rsnapshot plugin to create automatically snapshots from this remote (but locally mounted) drive.

I now tried the same with omv3.x and it does not work out the way I expected.
I can mount the smb share with the remote share plugin locally (somewhere to /srv/UID-123456789, probably the standard path)
But my rsnapshot setup does only create local empty folders (hourly.0, hourly.1, ...) on my local disk even thought the remote share is proper mounted and filled with data.

I now wonder what's going on here and if something changed between my versions of omv or if I just missing out something?

Also I know want to know what is the best practise (web gui wise) to do this with a clean omv landscape. My plan for a new (old keeps in place) setup is with two omv machines. One running as master and the other as slave, It should do excactly the same: mounting the remote share from omv-master and locally (omv-slave) create snapshots. I don't want to sync snapshots because it creates far more traffic in the network and also omv-master has only a small SSD where omv-slave has a big HDD.

I hope everything is clear

Hello,

I tracked it down.

The big difference is in the remote share plugin. For me it looks like a completely different plugin for omv 2.x vs omv 3.x. And it got worse for me

Problem is that the 'new' version in omv 3.x (beside that it now only supports smb/nfs and no more webdav etc.) does not 'register' the remote shares as shared folders. Therefor it's not possible to use rsnapshot because I can't chose the source folder (because it needs to be a shared folder). The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:

Failed to set file mode to '42775' for '/srv/xyz123456-1234-1234-1234-abcdefg1234/Share'

Even though the 'create window' stays open which would indicate that the creation of the shared folder failed but it got created anyway.

More errors to come when opening the rsnapshot tab:

Failed to execute XPath query '//system/shares/sharedfolder[uuid='zyx87654321-4321-4321-4321-gfedcba4321']'.

I guess it's because omv tries to manipulate things in the remote share. I don't even want that to happen. I just want to have the remote share mounted (maybe even read only) and create snapshots from it to the local disk.

The really bad thing about this. I just deployed a setup with two omv nas boxes and the snapshotting (which should be the remote backup and archive) does not work

The source is a NAS with SSD and limited space (500GB). Therefore it should just provide data shares. The target is a NAS with HDD and space (3TB) for the snapshots.

What to do?

Zitat von ryecoaaron

It is possible to use rsnapshot (although it is kind of dangerous since rsnapshot uses hard links which not all remote filesystems support properly) once you have created a sharedfolder using the remote mount.

The source and target is a OMV 3.0.99 both using ext4 as file system.

On my 'old' setup I have OMV 2.x running this (from a remote smb share) since many years without any problems. I also have a webdav mounted from which I do local rsnapshotting. Works wonderful and reliable.

Zitat von ryecoaaron

It doesn't mount the remote mount. It just creates the disk object. Once you create a shared folder, a mount is created. No tricks needed. If you get an error, then the setup/permissions are wrong and the OMV system can't create a folder on the remote share.

Actually I can create a share >inside< the remote share. But I want to do rsnapshot with the share itself.

I just created a new share on the source nas using the root of the disk (which is including the shares I want to do snapshotting). But it didn't work out. When I mount it on the target nas and create the rsnapshot job an hit start I get the little 'console' window with a overlay waiting (or something like this) followed by a completely white/empty window with a button 'close'. I tried couple of times but it's always empty and no error message is thrown.

The other thing I tried is to create a shared folder on the 'remote mount'/disk without creating a new folder inside but rather use '/' of the disk. Here I also get the error:

Failed to set file mode to '42775' ...

Zitat von ryecoaaron

You are the first to want this type of setup. Unfortunately, the plugin doesn't support this and would require porting the old plugin (I'm not going to do that) in order to do what you want. What I do want to do (someday) is change the rsnapshot plugin to allow remote hosts for the source but this would use the rsync protocol. Until then you can mount your remote shares manually and run rsnapshot manually or stay on OMV 2.x.

First and hopefully not the last! For me this setup has a lot of advantages:

• Extremely low on network traffic (just transferring the delta files which are changed or new on the source nas)
  
  
  • Therefor it's very efficient because uses less runtime transferring big amounts of data...
• Extremely low power (using to arm boxes with gigabit with far enough horse power)
• Extremely cheap (did I say it uses arm SoCs?)
• Easy 'decentralized' Backup (for example one nas in the basement other in the second floor)
• Easy archiving of the data (Thru snappshotting..)
• Easy going with two one-bay-nas but far more benefits than a two-bay-nas
• Taking advantages of different storage types: fast where necessary and cheap where possible (SSD to serve clients, HDD for backup and archive)
• Having a quick disaster recovery in case of emergency (using same hardware. Through easiness of omv I just need to create a share and activate it on the backup nas to be back on track)
• Many many more...

Anyway I can't go with OMV 2.x anymore because things are in place and in 2018 it's anyway to late for it. I rather need to go another way. If I read your post I now need to do to both manually: The mounting of the shares and the rsnapshot? I never did this and I'm used to the gui so this is rather a ruff edge.
What would be other possibilities with gui? Would it make sense to clone (rsync) the share from the source nas to the target nas and then create 'local' snapshots from this? Is this still possible?
Or any other ideas?

Thank's for your explanations by the way! Hope that omv get's this feature back in the future! It's a little regression for me

Zitat von ryecoaaron

You are probably the first because most people don't do this with samba shares which necessitates the remote mount plugin doing weird things. I understand the benefits but you need to do it a different way.

And I even added some more just this minute

Zitat von ryecoaaron

That is actually what I do on OMV 4.x. It requires more storage but it is worth it. Very reliable and eliminates samba.
I will try to look at the rsnapshot plugin to allow it to use a remote server for the source. Then you could run rsnapshot on the target server without creating the extra space requirement.

Great! I will go this way for now! Good that you have it running proper already. Any special tips how to configure the rsync? I think the source should do the rsync server an the target is playing client and is pulling the things? The standard settings are for good or any suggestions to chance them?

Pimping the rsnapshot plugin that it's possible to grab from the other nas would be awesome. Would be even better than my old omv2 setup

Zitat von ryecoaaron

For no particular reason, I am doing a push rsync from the source to the target. So, the target has the server modules enabled. Nothing else special.

I'm doing the same now with two OMV machines. Works without problems since over a month.

Anyway I need to come back here because I have almost the same scenario now but with a little detail which makes the solution in this thread not possible.

Difference is that there is not two but just one OMV server and the server which is serving the smb share is a windows machine. I'm almost breaking my head how to create a backup/snapshots of this share on the omv server (omv v3).

Zitat von ryecoaaron

Mount the smb share on your OMV system with the remotemount plugin. Then do an rsync/rsnapshot from a shared folder created using the remotemount mount.

This worked in omv v2 for me but as described in the first post it didn't worked for me in omv v3. Is there something I missed? A update from remotemount or something else?

Zitat von ryecoaaron

I use this technique on a server. I know it works. /srv/UID-123456789 may have data in it but when you create a shared folder (used by rsnapshot), it creates a subfolder named the same as the sharedfolder name. Is there data in that subfolder?

When I mount the (windows) smb share with remotemount in my omv (v3.0.99) server and then try to create a sharedfolder from the new mountpoint it just throw's errors like hell.

This is my conclusion:

Zitat von Wasn

Problem is that the 'new' version in omv 3.x (beside that it now only supports smb/nfs and no more webdav etc.) does not 'register' the remote shares as shared folders. Therefor it's not possible to use rsnapshot because I can't chose the source folder (because it needs to be a shared folder). The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:

Failed to set file mode to '42775' for '/srv/xyz123456-1234-1234-1234-abcdefg1234/Share'
Even though the 'create window' stays open which would indicate that the creation of the shared folder failed but it got created anyway.

More errors to come when opening the rsnapshot tab:

Failed to execute XPath query '//system/shares/sharedfolder[uuid='zyx87654321-4321-4321-4321-gfedcba4321']'.

I guess it's because omv tries to manipulate things in the remote share. I don't even want that to happen. I just want to have the remote share mounted (maybe even read only) and create snapshots from it to the local disk.

Alles anzeigen

What I want to achieve is a "simple" backup/snapshotting from the share without touching it (read-only style).

Zitat von ryecoaaron

I know it works. /srv/UID-123456789 may have data in it but when you create a shared folder (used by rsnapshot), it creates a subfolder named the same as the sharedfolder name. Is there data in that subfolder?

Just to be sure: For rsnapshot I need a shared folder. And when I tried to make a shared folder with the mountpoint created by remotemount I get this errors reported:

Zitat von Wasn

I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:

Failed to set file mode to '42775' for '/srv/xyz123456-1234-1234-1234-abcdefg1234/Share'

Even though the 'create window' stays open which would indicate that the creation of the shared folder failed but it got created anyway.

More errors to come when opening the rsnapshot tab:

Failed to execute XPath query '//system/shares/sharedfolder[uuid='zyx87654321-4321-4321-4321-gfedcba4321']'.

Alles anzeigen

What do I do different than you that it doesn't work for me?