Remote Folder sync with to OMV NAS behind Pfsense firewall

    Hello everybody, I need to securely backup my photo folder, which in on a OMV NAS located in my office behind a pfSense firewall to another folder located in my OMV NAS at home, behind another pfSense firewall.
    Could someone explain to a newbe (myself!) which in in theory the best way to do that?
    I've read something in the Web and mount an NFS REMOTE SHARE (with the homonymous plugin) in my home OMV NAS and rsync it with the local (office) share could be a goog solution, but I don't understand the pqassages to do that, including the pfSense configuration.
    Again, I don't understand the nest way to make it secure (ssh, vnp tunnel, etc....)
    In there a patient brother who can help me?
    Thanks in advice
    Massimo

    If you can wait a little longer for a proper response, I can probably help you here... as I'm looking to backup my OMV NAS through my pfsense firewall to a RasPi running OMV at another physical location... so quite similar to your setup :)


    But the hardware only arrived this week, so I've not proved this at all :)


    My planned layout will be:


    OMV (on a PC) --> pfsense --> Internet --> OMV (on a Ras Pi2)


    This is using OMV 2.2.11 with pfsense 2.3.2


    My plan is to either use the rsync daemon (preferred) or perhaps the syncthing plugin (I'm not sure about Syncthing's encryption - yet).


    Both OMV devices will have the fail2ban plugin enabled on SSH & Apache - and if needed - on the rsync daemon port


    rsync is my preferred first choice because it's a Linux standard tool... so even if OMV stops working, I could continue with the base OS tools.


    Both syncthing and rsync will only transfer the changes in files - so (for example) if you edited the meta data on a 40MB photo, it would only transfer the metadata (~kB), not the entire photo again (~MB)


    To be clear here - my use is a one-way mirror (see the arrows above)... so again, rsync works well here, but if you wanted a 2-way sync, then you would need to carefully think about how that would work... or use Syncthing (that handles 2-way updates)


    Either way - I think that the transfer should be over an SSH tunnel between the sites, initiated by OMV, passing through the firewall and terminating at the other OMV.


    So, as I understand your situation your pfsense box at home would need to have a firewall rule to allow the SSH connection "IN" to your NAS. I'm presuming that you don't allow SSH to the firewall from the internet, but even so, you might want to consider using NAT to change the port from the standard SSH one (22) to another one... ie 2222 (a bad example)


    So, in pfsense, you would need to go to Firewall->NAT->Port Forward and allow 2222 on your WAN interface to SSH on your NAS.


    Now, that's as far as I know (so far).


    I'm expecting that I will need to generate SSH keys so that both NAS can connect to each other over SSH without using a password and then rsync can communicate through the SSH tunnel (ie using the default port 873)


    That's the part that I have not reached yet... :)


    If anyone else has a better solution, I'm all ears...

    Hi, thanks for your answer.
    So far I opened a port on the first OMV (let's call it "server"), where is located my photo "original" folder: I chose an unconventional port (ie 7878 or something like that) and after with a NAT rule I redirected the incoming traffic to the 22 port or OMV, which has the "root" access and the ssh tunneling enabled. So, actually, I can actuallty reach my nas from outside. Yhe question is now HOW to transfer data. I'm a newbe so let me say stupidities, as I understoon reading over tue Net.
    1) Rsync demon sounds insicure, 'cause not allows encrypted transmission (is that true), unless it works inside a VPN tunnel;
    2) I'd prefere to configure a remote folder with the utility, pointing a "server" NFS share (1st question: HOW DO IT?) and configure an SSH rsync job (2nd question: HOW?).
    But this is theory, and my attempts of doing it failed so far... :cursing::cursing::cursing:
    Syncthing could make the job, but I can't manage working it: after hours and hours of sincronization, it quit working whitout apparent reason and the folder renains "out of sync forever ... :cursing::cursing::cursing:
    Bittorrent is even more malicious: after few hours, the server GUI (I had and "only read" secret on the other side) says that it can't recognise the target folder... :cursing::cursing::cursing:
    HATE THEM BOTH: the shoud worh "out of the box" in UnPn... ?(?(?(
    And guess what: suddenly my 8GB ssd, on which I have OMV system installed, became suddenly FULL, blocking some jobs. After a good hour of searching it came out that was full of syncthing logs (5 Gb!!!), while the plugin was uninstalled days before!!!


    Appreciate your help my friend: let me know your progessions.


    Bye


    Massimo

    OK... so my findings so far...


    (Again, I am happy for anyone else to correct me here... :) )


    So, I'm currently presuming that I need a specific user to SSH from one OMV NAS to the other, so I have created a specific "rsync mirroring" user for this.


    I then need to generate the private / public keys, which are stored in that user's home folder... so it took me a minute to understand that OMV doesn't have a "/home" folder by default, so I needed to setup a shared folder to locate the "/home" folder, then enable home folders under Access Rights Management -> User -> Settings.


    OK, so at this point I use the terminal and SSH from each OMV to the other which adds the "other" OMV to the ~/.ssh/known_hosts file


    Then I created the private / public keys (on both OMV) using ssh-keygen and copied the public key (.pub) to the "rsync mirroring" user on the "other" OMV using ssh-copy-id


    Testing, I can SSH from each OMV to the other without using a password now... that means that a password does not need to be sent over the internet.


    It also means that the SSH tunnel is working through the pfsense firewall, including NAT, etc.


    Now, the part that I'm stuck at, is rsync... or more specifically, OMV's implementation of rsync


    OMV has 2x abilities with rsync: "jobs" and "server"


    "Jobs" are simply scheduled rsync commands - but they run as root
    "Server" is an rsync daemon, which requires a module to be configured (which can only rsync an entire shared folder) - and these can run as different users, but only using the rsync protocol, not over SSH (as far as I understand)


    :/


    So... I think that I need a "job" that runs as the "rsync mirroring" user, not root. ?(


    And this is where I have reached (so far...)


    I'm wondering whether I need to create a Scheduled Job (under the System menu tree, NOT an Rsync job) which can run as the "rsync mirroring" user... and then that does the actual rsync... :huh:


    Seems a bit weird, hence I'm not sure I'm going about this the correct way... but I do not want my data being transferred via the root user...


    I'll report back what I find...

    Just a quick note about Syncthing...


    I don't know if you use Synthing-GTK or the Syncthing WebGUI, but I found that it took a while to understand how it worked - including crazy things like the "hidden" .stversions folder that can become huge if you have file versioning enabled...


    So, I have it running at home with 3x laptops, 2x PC a Raspberry Pi and my OMV NAS... I just started slowly


    But... that's a separate subject and not about this thread ;)

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!

Benutzerkonto erstellen Anmelden