Best practise for backing-up/snapshotting a remote smb share to omv

    • OMV 3.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Best practise for backing-up/snapshotting a remote smb share to omv

      Hey there,

      I'm running a couple of omv setups and struggeling right now a little bit.

      I used to (omv2.x) use the remote share plugin to mount a smb (and webdav) share localy in omv.
      Then I used the rsnapshot plugin to create automatically snapshots from this remote (but locally mounted) drive.

      I now tried the same with omv3.x and it does not work out the way I expected.
      I can mount the smb share with the remote share plugin locally (somewhere to /srv/UID-123456789, probably the standard path)
      But my rsnapshot setup does only create local empty folders (hourly.0, hourly.1, ...) on my local disk even thought the remote share is proper mounted and filled with data.

      I now wonder what's going on here and if something changed between my versions of omv or if I just missing out something?



      Also I know want to know what is the best practise (web gui wise) to do this with a clean omv landscape. My plan for a new (old keeps in place) setup is with two omv machines. One running as master and the other as slave, It should do excactly the same: mounting the remote share from omv-master and locally (omv-slave) create snapshots. I don't want to sync snapshots because it creates far more traffic in the network and also omv-master has only a small SSD where omv-slave has a big HDD.

      I hope everything is clear :D
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/

      The post was edited 1 time, last by Wasn ().

    • Hello,

      I tracked it down.

      The big difference is in the remote share plugin. For me it looks like a completely different plugin for omv 2.x vs omv 3.x
      . And it got worse for me :/

      Problem is that the 'new' version in omv 3.x (beside that it now only supports smb/nfs and no more webdav etc.) does not 'register' the remote shares as shared folders. Therefor it's not possible to use rsnapshot because I can't chose the source folder (because it needs to be a shared folder). The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:

      Source Code

      1. Failed to set file mode to '42775' for '/srv/xyz123456-1234-1234-1234-abcdefg1234/Share'

      Even though the 'create window' stays open which would indicate that the creation of the shared folder failed but it got created anyway.

      More errors to come when opening the rsnapshot tab:

      Source Code

      1. Failed to execute XPath query '//system/shares/sharedfolder[uuid='zyx87654321-4321-4321-4321-gfedcba4321']'.


      I guess it's because omv tries to manipulate things in the remote share. I don't even want that to happen. I just want to have the remote share mounted (maybe even read only) and create snapshots from it to the local disk.

      The really bad thing about this. I just deployed a setup with two omv nas boxes and the snapshotting (which should be the remote backup and archive) does not work ;(

      The source is a NAS with SSD and limited space (500GB). Therefore it should just provide data shares. The target is a NAS with HDD and space (3TB) for the snapshots.

      What to do?
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/
    • Wasn wrote:

      The big difference is in the remote share plugin. For me it looks like a completely different plugin for omv 2.x vs omv 3.x.
      That is because it was re-written completely. Hence why it has a different name - remoteshare vs remotemount

      Wasn wrote:

      Problem is that the 'new' version in omv 3.x (beside that it now only supports smb/nfs and no more webdav etc.) does not 'register' the remote shares as shared folders.
      It isn't a problem. It is the correct method in OMV. Remotemount doesn't bind mount to an existing sharedfolder but it does create a device that you can use to create shared folders.

      Wasn wrote:

      Therefor it's not possible to use rsnapshot because I can't chose the source folder (because it needs to be a shared folder). The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:
      It is possible to use rsnapshot (although it is kind of dangerous since rsnapshot uses hard links which not all remote filesystems support properly) once you have created a sharedfolder using the remote mount.

      Wasn wrote:

      The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:
      It doesn't mount the remote mount. It just creates the disk object. Once you create a shared folder, a mount is created. No tricks needed. If you get an error, then the setup/permissions are wrong and the OMV system can't create a folder on the remote share.

      Wasn wrote:

      The really bad thing about this. I just deployed a setup with two omv nas boxes and the snapshotting (which should be the remote backup and archive) does not work

      The source is a NAS with SSD and limited space (500GB). Therefore it should just provide data shares. The target is a NAS with HDD and space (3TB) for the snapshots.

      What to do?
      You are the first to want this type of setup. Unfortunately, the plugin doesn't support this and would require porting the old plugin (I'm not going to do that) in order to do what you want. What I do want to do (someday) is change the rsnapshot plugin to allow remote hosts for the source but this would use the rsync protocol. Until then you can mount your remote shares manually and run rsnapshot manually or stay on OMV 2.x.
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Wasn wrote:

      Therefor it's not possible to use rsnapshot because I can't chose the source folder (because it needs to be a shared folder). The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:
      It is possible to use rsnapshot (although it is kind of dangerous since rsnapshot uses hard links which not all remote filesystems support properly) once you have created a sharedfolder using the remote mount.
      The source and target is a OMV 3.0.99 both using ext4 as file system.

      On my 'old' setup I have OMV 2.x running this (from a remote smb share) since many years without any problems. I also have a webdav mounted from which I do local rsnapshotting. Works wonderful and reliable.


      ryecoaaron wrote:

      Wasn wrote:

      The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:
      It doesn't mount the remote mount. It just creates the disk object. Once you create a shared folder, a mount is created. No tricks needed. If you get an error, then the setup/permissions are wrong and the OMV system can't create a folder on the remote share.

      Actually I can create a share >inside< the remote share. But I want to do rsnapshot with the share itself.

      I just created a new share on the source nas using the root of the disk (which is including the shares I want to do snapshotting). But it didn't work out. When I mount it on the target nas and create the rsnapshot job an hit start I get the little 'console' window with a overlay waiting (or something like this) followed by a completely white/empty window with a button 'close'. I tried couple of times but it's always empty and no error message is thrown.

      The other thing I tried is to create a shared folder on the 'remote mount'/disk without creating a new folder inside but rather use '/' of the disk. Here I also get the error:

      Source Code

      1. Failed to set file mode to '42775' ...




      ryecoaaron wrote:

      Wasn wrote:

      The really bad thing about this. I just deployed a setup with two omv nas boxes and the snapshotting (which should be the remote backup and archive) does not work

      The source is a NAS with SSD and limited space (500GB). Therefore it should just provide data shares. The target is a NAS with HDD and space (3TB) for the snapshots.

      What to do?
      You are the first to want this type of setup. Unfortunately, the plugin doesn't support this and would require porting the old plugin (I'm not going to do that) in order to do what you want. What I do want to do (someday) is change the rsnapshot plugin to allow remote hosts for the source but this would use the rsync protocol. Until then you can mount your remote shares manually and run rsnapshot manually or stay on OMV 2.x.

      First and hopefully not the last! For me this setup has a lot of advantages:
      • Extremely low on network traffic (just transferring the delta files which are changed or new on the source nas)
        • Therefor it's very efficient because uses less runtime transferring big amounts of data...
      • Extremely low power (using to arm boxes with gigabit with far enough horse power)
      • Extremely cheap (did I say it uses arm SoCs?)
      • Easy 'decentralized' Backup (for example one nas in the basement other in the second floor)
      • Easy archiving of the data (Thru snappshotting..)
      • Easy going with two one-bay-nas but far more benefits than a two-bay-nas
      • Taking advantages of different storage types: fast where necessary and cheap where possible (SSD to serve clients, HDD for backup and archive)
      • Having a quick disaster recovery in case of emergency (using same hardware. Through easiness of omv I just need to create a share and activate it on the backup nas to be back on track)
      • Many many more...
      Anyway I can't go with OMV 2.x anymore because things are in place and in 2018 it's anyway to late for it. I rather need to go another way. If I read your post I now need to do to both manually: The mounting of the shares and the rsnapshot? I never did this and I'm used to the gui so this is rather a ruff edge. ||
      What would be other possibilities with gui? Would it make sense to clone (rsync) the share from the source nas to the target nas and then create 'local' snapshots from this? Is this still possible?
      Or any other ideas? ?(

      Thank's for your explanations by the way! Hope that omv get's this feature back in the future! It's a little regression for me :whistling:
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/

      The post was edited 1 time, last by Wasn ().

    • Wasn wrote:

      First and hopefully not the last! For me this setup has a lot of advantages
      You are probably the first because most people don't do this with samba shares which necessitates the remote mount plugin doing weird things. I understand the benefits but you need to do it a different way.

      Wasn wrote:

      If I read your post I now need to do to both manually: The mounting of the shares and the rsnapshot? I never did this and I'm used to the gui so this is rather a ruff edge
      Yes, IF you want to keep doing it the same way.

      Wasn wrote:

      What would be other possibilities with gui? Would it make sense to clone (rsync) the share from the source nas to the target nas and then create 'local' snapshots from this? Is this still possible?
      That is actually what I do on OMV 4.x. It requires more storage but it is worth it. Very reliable and eliminates samba.

      I will try to look at the rsnapshot plugin to allow it to use a remote server for the source. Then you could run rsnapshot on the target server without creating the extra space requirement.
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Wasn wrote:

      First and hopefully not the last! For me this setup has a lot of advantages
      You are probably the first because most people don't do this with samba shares which necessitates the remote mount plugin doing weird things. I understand the benefits but you need to do it a different way.
      And I even added some more just this minute ;)


      ryecoaaron wrote:

      Wasn wrote:

      What would be other possibilities with gui? Would it make sense to clone (rsync) the share from the source nas to the target nas and then create 'local' snapshots from this? Is this still possible?
      That is actually what I do on OMV 4.x. It requires more storage but it is worth it. Very reliable and eliminates samba.
      I will try to look at the rsnapshot plugin to allow it to use a remote server for the source. Then you could run rsnapshot on the target server without creating the extra space requirement.
      Great! I will go this way for now! Good that you have it running proper already. Any special tips how to configure the rsync? I think the source should do the rsync server an the target is playing client and is pulling the things? The standard settings are for good or any suggestions to chance them?

      Pimping the rsnapshot plugin that it's possible to grab from the other nas would be awesome. Would be even better than my old omv2 setup :thumbsup:
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/
    • Wasn wrote:

      Any special tips how to configure the rsync? I think the source should do the rsync server an the target is playing client and is pulling the things? The standard settings are for good or any suggestions to chance them?
      For no particular reason, I am doing a push rsync from the source to the target. So, the target has the server modules enabled. Nothing else special.
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Wasn wrote:

      Any special tips how to configure the rsync? I think the source should do the rsync server an the target is playing client and is pulling the things? The standard settings are for good or any suggestions to chance them?
      For no particular reason, I am doing a push rsync from the source to the target. So, the target has the server modules enabled. Nothing else special.
      I'm doing the same now with two OMV machines. Works without problems since over a month. :thumbup:


      Anyway I need to come back here because I have almost the same scenario now but with a little detail which makes the solution in this thread not possible.

      Difference is that there is not two but just one OMV server and the server which is serving the smb share is a windows machine. I'm almost breaking my head how to create a backup/snapshots of this share on the omv server (omv v3). <X
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/
    • Wasn wrote:

      Difference is that there is not two but just one OMV server and the server which is serving the smb share is a windows machine. I'm almost breaking my head how to create a backup/snapshots of this share on the omv server (omv v3).
      Mount the smb share on your OMV system with the remotemount plugin. Then do an rsync/rsnapshot from a shared folder created using the remotemount mount.
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Wasn wrote:

      Difference is that there is not two but just one OMV server and the server which is serving the smb share is a windows machine. I'm almost breaking my head how to create a backup/snapshots of this share on the omv server (omv v3).
      Mount the smb share on your OMV system with the remotemount plugin. Then do an rsync/rsnapshot from a shared folder created using the remotemount mount.
      This worked in omv v2 for me but as described in the first post it didn't worked for me in omv v3. Is there something I missed? A update from remotemount or something else?
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/
    • Wasn wrote:

      This worked in omv v2 for me but as described in the first post it didn't worked for me in omv v3.
      I didn't read the entire thread.

      Wasn wrote:

      Is there something I missed? A update from remotemount or something else?
      I use this technique on a server. I know it works. /srv/UID-123456789 may have data in it but when you create a shared folder (used by rsnapshot), it creates a subfolder named the same as the sharedfolder name. Is there data in that subfolder?
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:


      Wasn wrote:

      Is there something I missed? A update from remotemount or something else?
      I use this technique on a server. I know it works. /srv/UID-123456789 may have data in it but when you create a shared folder (used by rsnapshot), it creates a subfolder named the same as the sharedfolder name. Is there data in that subfolder?
      When I mount the (windows) smb share with remotemount in my omv (v3.0.99) server and then try to create a sharedfolder from the new mountpoint it just throw's errors like hell.

      This is my conclusion:

      Wasn wrote:

      Problem is that the 'new' version in omv 3.x (beside that it now only supports smb/nfs and no more webdav etc.) does not 'register' the remote shares as shared folders. Therefor it's not possible to use rsnapshot because I can't chose the source folder (because it needs to be a shared folder). The 'new' plugin in omv 3.x rather mounts the remote share as a disk (I'm not sure if its a bug or a feature?).
      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:


      Failed to set file mode to '42775' for '/srv/xyz123456-1234-1234-1234-abcdefg1234/Share'
      Even though the 'create window' stays open which would indicate that the creation of the shared folder failed but it got created anyway.

      More errors to come when opening the rsnapshot tab:


      Failed to execute XPath query '//system/shares/sharedfolder[uuid='zyx87654321-4321-4321-4321-gfedcba4321']'.

      I guess it's because omv tries to manipulate things in the remote share. I don't even want that to happen. I just want to have the remote share mounted (maybe even read only) and create snapshots from it to the local disk.

      What I want to achieve is a "simple" backup/snapshotting from the share without touching it (read-only style).
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/
    • Wasn wrote:

      When I mount the (windows) smb share with remotemount in my omv (v3.0.99) server and then try to create a sharedfolder from the new mountpoint it just throw's errors like hell
      Then you permissions on the remote server and/or the mount options are wrong.

      Wasn wrote:

      This is my conclusion:
      Most of that is wrong except the part about treats the remote share as a disk instead of bind mounting it to an existing shared folder like the OMV 2.x version did (which is the wrong way to do it). Creating the shared folder will fail if the smb share is read only and the folder named after the shared folder name doesn't exist.

      Wasn wrote:

      What I want to achieve is a "simple" backup/snapshotting from the share without touching it (read-only style).
      Everything in OMV uses shared folders. So, your "simple" idea is difficult to implement. On my Windows machines, I have scheduled task that copies to an OMV samba share. From there, that shared folder is setup in rsnapshot job. Does that create an extra copy of data on the OMV box? Yes but I don't mind that extra backup copy.
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      I know it works. /srv/UID-123456789 may have data in it but when you create a shared folder (used by rsnapshot), it creates a subfolder named the same as the sharedfolder name. Is there data in that subfolder?
      Just to be sure: For rsnapshot I need a shared folder. And when I tried to make a shared folder with the mountpoint created by remotemount I get this errors reported:

      Wasn wrote:

      I tried to trick the system in creating a shared folder on the disk (which is the remote share) but than it's throwing errors in the shared folder tab when creating the share:


      Failed to set file mode to '42775' for '/srv/xyz123456-1234-1234-1234-abcdefg1234/Share'


      Even though the 'create window' stays open which would indicate that the creation of the shared folder failed but it got created anyway.

      More errors to come when opening the rsnapshot tab:


      Failed to execute XPath query '//system/shares/sharedfolder[uuid='zyx87654321-4321-4321-4321-gfedcba4321']'.

      What do I do different than you that it doesn't work for me?
      "This post will remain inaccessible for others until approved by a censor." - George Orwell X/

      The post was edited 1 time, last by Wasn ().

    • Is the second post meant to point out the words I am missing??

      Wasn wrote:

      What do I do different than you that it doesn't work for me?
      No idea. I have no idea how your samba share is setup and I don't how your remotemount setup is setup. This is the biggest pain about this plugin. The plugin doesn't control half of the equation. I'm sorry if you want to lock down the samba share and the shared folder creation process doesn't work. I don't know how to avoid that. If you want to manually create the setup that the old remote share process creates, just make a shared folder, add a cifs fstab entry (outside of the OMV lines) and bind mount it to the newly created shared folder. Then you can use that shared folder in rsnapshot as a source.
      omv 4.1.7 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!