Sharing OMV shares over the Internet

    • OMV 3.x
    • Sharing OMV shares over the Internet

      Hi everyone,

      I've got OMV all set up and configured with various shares/file permissions/users. I want users to be able to map some shares as network drives on another network, to be in use 24/7.

      I have samba all up and running on my local network, works like a charm. Unfortunately, my ISP blocks any incoming/(and supposedly outgoing) port 445 connections, and all incoming port 137-139 connections. I'm aware of Samba's security issues, but if a modern samba share is configured properly, my understanding is that it is reasonably secure. So, I configured Samba to use port 2111 and forwarded it on my router, and sure enough, samba works over the internet, but only for mac/linux! When a client on mac/linux tries to connect, they specify the port with a colon, such as \\65.55.55.55:2111\samba\share. Windows has no feature to map a network drive over a different port, 445 is essentially "hard coded in". This removes my ability to set up samba such that other windows clients on another net can connect.

      Which brings me to this post, has anyone been able to figure out how to map a network drive and specify the port in windows?!

      Assuming this isn't possible, I tried to set up NFS but cannot for the life of me figure out how to set up proper username/password configurations for NFS shares, which worked on samba out of the box in OMV.

      On windows 10 pro with the NFS services installed, the mount command has a flag for username/password (-u:username -p:password).
      But nothing seems to work. I keep reading something about UID/GUID for NFS, but see no options for this anywhere in the UI, nor how this could translate to a reasonable user/password for clients to use. Any tips of setting up NFS would be appreciated. The main advantage of NFS is that its ports wont be blocked by my ISP, so this could be a reasonable alternative to SMB.

      As it stands, I cannot figure out how to set up NFS shares with a username/password that can easily be mounted by clients, any help would be appreciated.


      Reading more, I have heard of 3 alternative options for getting SMB to work over the internet:

      1. OpenVPN.
      Supposedly I can setup a VPN so that others can VPN into my box. This VPN session would have to be active, and running on my omv box. Could OpenVPN support 10+ clients all connected to my network at once, for the sole purpose of accessing the SMB shares? Is this user friendly? Can clients set it up and forget about it, as the openVPN program will automatically reconnect them to my network, and SMB will work out? Or will openVPN take a bit to connect, and windows network drive throw a bunch of errors since the drive isn't reachable immediately whenever the client restarts their computer or temporarily disconnects from the network?

      Secondly, if this VPN is set up, is there any way so that they can use the vpn to ONLY access SMB through the VPN, and aren't tunneling all their traffic through my network, which would vastly increase my data usages and slow down my network unnecessarily.

      2. SSH.
      I've heard allowing for an SSH tunnel can somehow give clients access to my local network, and mount SMB shares that way. No clue how this is possible, or if its easy to set up permanently on a windows system. Any explanation on this would be great.

      3. DynDns/NoIp/some other DNS equivalent.
      Can I set up a DNS that accepts connections from other networks through port 445, and points to my SMB partition over port 2111 (instead of 445, since my ISP blocks it). This would allow clients off the network to map the drive in windows normally, which would send an SMB request on port 445, and the DNS would intercept this traffic and reroute it to 2111 on my end, bypassing both my ISPs port block of 455, as well as the windows' clients inability to specify a port other than 445.




      tl;dr Need to share various shares with 10+ users over the internet, have gigabit internet so this shouldn't be a problem. ISP blocks port 445, what is a user-friendly solution to map a network drive to be used 24/7?

      Sorry for the long post, and thank you so much to anyone willing to help and give insight!
    • I got to the part where you shared samba over the internet and stopped reading. Please stop now and do not share samba (or nfs) over the internet. They are NOT secure enough.

      1 - This is your best bet. This is how it is done in the enterprise. I don't use it myself though.

      2 - I wouldn't use ssh to try to tunnel samba.

      3 - no no no

      You could also look into webdav, syncthing, owncloud, etc.
      omv 4.1.12 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      I got to the part where you shared samba over the internet and stopped reading. Please stop now and do not share samba (or nfs) over the internet. They are NOT secure enough.

      1 - This is your best bet. This is how it is done in the enterprise. I don't use it myself though.

      2 - I wouldn't use ssh to try to tunnel samba.

      3 - no no no

      You could also look into webdav, syncthing, owncloud, etc.
      I'll look into OpenVPN as a means for users to have a link to our local network 24/7. Any recommended guides/links for using OpenVPN with OMV? I see there's a plugin but it seems like many options still need to be manually configured.

      Also, I hear people say SMB is very insecure, but I know of a lot of Universities/Corporations that have clients map network drives through SMB. My understanding is that modern SMB, if configured properly, is secure. Are there unpatched bugs and vulnerabilities in modern SMB that are unavoidable?

      Really appreciate your help :)

      Nibb31 wrote:

      I would question why you want direct SMB access over the internet instead of a syncing or streaming solution.
      The partition being mounted is over 50tb, and users need access to all of the data and cannot possibly manage storing it all or syncing it all. They all pay for high-bandwidth internet (200mb/s+) so throughput shouldn't be an issu
      We've assessed many of the typical syncing tools and even FTP, but mounting as a network drive would be the most convenient option possible
    • aurora wrote:

      They all pay for high-bandwidth internet (200mb/s+) so throughput shouldn't be an issue
      Throughput is never the issue which such setups but latency if they're not very close to you -- you can check this with ping latency.

      Small try to explain: Packets between server and client need an ACK that they arrived at the other side, SMB and other NAS protocols are optimized for LAN usage (high bandwidth and also very low latency), while WAN connections might have an ok-ish bandwidth but most probably a latency too high. It's pretty normal with a WAN connection capable of 200 Mbps a single client server transfer being limited to maybe 5 MBps due to this problem. Folder enumeration will take ages of course, the only thing that might get faster these days is a copy made with Windows Explorer if clients use Windows 7 or above (since the SMB redirector then establishes up to 8 connections in parallel)

      Do a web search for 'riverbed wan acceleration cifs', try to understand why companies invest $50k into such WAN accelerators and then simply give it a try with OpenVPN (on a dedicated box of course with all 'low latency' optimizations possible). And yes, OpenVPN settings allow that only the specific connections needed to access your server go through the VPN and no, most probably will performance suck. I can tell this since I code custom data sync solutions for customers that already wasted lots of money failing with 'LAN protocol used over WAN' attempts.

      If all participants are rather close together (check ping roundtrip times) then it might work ok-ish but I doubt it.
    • tkaiser wrote:

      aurora wrote:

      They all pay for high-bandwidth internet (200mb/s+) so throughput shouldn't be an issue
      Throughput is never the issue which such setups but latency if they're not very close to you -- you can check this with ping latency.
      Small try to explain: Packets between server and client need an ACK that they arrived at the other side, SMB and other NAS protocols are optimized for LAN usage (high bandwidth and also very low latency), while WAN connections might have an ok-ish bandwidth but most probably a latency too high. It's pretty normal with a WAN connection capable of 200 Mbps a single client server transfer being limited to maybe 5 MBps due to this problem. Folder enumeration will take ages of course, the only thing that might get faster these days is a copy made with Windows Explorer if clients use Windows 7 or above (since the SMB redirector then establishes up to 8 connections in parallel)

      Do a web search for 'riverbed wan acceleration cifs', try to understand why companies invest $50k into such WAN accelerators and then simply give it a try with OpenVPN (on a dedicated box of course with all 'low latency' optimizations possible). And yes, OpenVPN settings allow that only the specific connections needed to access your server go through the VPN and no, most probably will performance suck. I can tell this since I code custom data sync solutions for customers that already wasted lots of money failing with 'LAN protocol used over WAN' attempts.

      If all participants are rather close together (check ping roundtrip times) then it might work ok-ish but I doubt it.
      I see, all the hosts are within 5-10 miles of the OMV but one, who lives roughly ~1500 miles away. The operations in general aren't latency intensive, but if bulk file transfers will be down to 40mb/s, this could be a bit of an issue. Fortunately, all clients are on Windows 10 pro to my knowledge.

      Are there any WAN protocols to map a network drive that don't require file sync to the clients' physical drives?
    • aurora wrote:

      The operations in general aren't latency intensive
      I think you missed the point. Maybe do a web search for 'bandwidth latency calculator'...

      And then do a web search for 'openvpn cifs optimization' and stuff like that. This is all not related to OMV at all, just how to solve the 'latency too high' problem with protocols that are optimized for LAN networks. And please think about again: Corporations pay happily insane amounts of money for 'WAN accelerators' that try to address the problem by caching locally as much as possible which of course won't work that great if you're talking about 50 TB :)

      aurora wrote:

      Are there any WAN protocols to map a network drive that don't require file sync to the clients' physical drives?
      If you call FTP/FTPS or SFTP (two totally different things) a WAN protocol then you might find some VFS layer functionality that make remote shares available so that applications can access stuff there directly (often implemented via FUSE -- no idea whether that's available on Windows, fortunately I really don't need to deal with Windows at all). But this doesn't solve the problem and the only attempts that work over larger distances (again: check ping roundtrip times, that's the only important number) are preparing/syncing data.

      Many customers use Helios Webshare for this but it's quite expensive and in the end just 'partial sync and resync' which has it's own problems.
    • I guess I'll chime in here a bit since I find this thread a bit interesting. I'm curious about this "Also, I hear people say SMB is very insecure, but I know of a lot of Universities/Corporations that have clients map network drives through SMB" because I know that if you're within the network, you should always be able to map a network drive. You said you know they are doing it that, but how are they doing it? From outside a vpn seems to be more reasonable or webdav as mentioned before. You did mention that there's a lot of data which is the reason you want to map the drive. Once you are logged in with a vpn you have access to the whole network and work just as you are there if I remember correctly. It doesn't matter where each person is at unless they have a really slow line.
    • Jaime wrote:

      Once you are logged in with a vpn you have access to the whole network and work just as you are there if I remember correctly.
      Sure. But the VPN can be setup to only allow access to this single server (to prevent opening up entire networks), some stuff that works on 'link local' layer like automatic announcement of file shares will not work through VPN connections (so a server won't show up automagically in the side bar but you have to know the address) and performance can be an issue as explained above depending on latency between client and server.
    • tkaiser wrote:

      And then do a web search for 'openvpn cifs optimization' and stuff like that. This is all not related to OMV at all, just how to solve the 'latency too high' problem with protocols that are optimized for LAN networks. And please think about again: Corporations pay happily insane amounts of money for 'WAN accelerators' that try to address the problem by caching locally as much as possible which of course won't work that great if you're talking about 50 TB
      I see, could you elaborate a bit more on why LAN protocols are so much slower than WAN protocls over the internet? Do LAN-based protocols operate on some different level of the OSI in how they handle networking?
      Clients using FTP to access my OMV are getting near 100% of their advertised speeds, and FTP uses TCP, although a TCP throughput calculator shows the theoretical limits should be much lower. I have some fundamental misunderstanding of LAN protocols and why they function so differently.

      tkaiser wrote:

      If you call FTP/FTPS or SFTP (two totally different things) a WAN protocol then you might find some VFS layer functionality that make remote shares available so that applications can access stuff there directly (often implemented via FUSE -- no idea whether that's available on Windows, fortunately I really don't need to deal with Windows at all). But this doesn't solve the problem and the only attempts that work over larger distances (again: check ping roundtrip times, that's the only important number) are preparing/syncing data.

      Many customers use Helios Webshare for this but it's quite expensive and in the end just 'partial sync and resync' which has it's own problems.
      I'll look into getting FUSE working. I've had to do it once before on mac/linux/windows, if I remember correctly, getting it working on Mac was a nightmare. There was some library necessary to mount it, and it was horribly out of date. required so much tweaking/reading github comments to get working on the latest two versions of OSX. Performance was lackluster, couldn't pull more than 2MB/s, while getting 60-70MB/s over FTP from the same server. Perhaps the server wasn't set up correctly?




      Jaime wrote:

      I guess I'll chime in here a bit since I find this thread a bit interesting. I'm curious about this "Also, I hear people say SMB is very insecure, but I know of a lot of Universities/Corporations that have clients map network drives through SMB" because I know that if you're within the network, you should always be able to map a network drive. You said you know they are doing it that, but how are they doing it? From outside a vpn seems to be more reasonable or webdav as mentioned before. You did mention that there's a lot of data which is the reason you want to map the drive. Once you are logged in with a vpn you have access to the whole network and work just as you are there if I remember correctly. It doesn't matter where each person is at unless they have a really slow line.
      These corporations/Universities don't have residential ISP plans that block port 445. They are free to open up port 445 for incoming connections, and viola, the network drive can be mapped over the internet. This is the same with email servers and SMTP servers, just forward the port in your router and they are freely accessible (just be ready for a potential security nightmare). Many residential ISPs block these ports to prevent their customers from hosting such services outside of their home. Partially due to security, partly to reduce their bandwidth costs, partly to prevent the issues it may cause that they eventually have to deal with, and of course, partly to encourage their customers to purchase a more expensive business plan.





      Jaime wrote:

      or webdav as mentioned before
      How is the performance of webdav? Looking through posts here, it looks like its not nearly as well supported as samba. I still need robust file permissions and user/pass authentication, which a post from 2015 says isn't supported yet?
      Could you link me to some further reading on how I can set up webdav with user/pass authentication and setting read/write permissions on a per-user and per-share basis?






      Currently doing work on setting up OpenVPN for my box. Will report on performance, Any recommended links you all have related to setting up openVPN/optimizing openVPN would be much appreciated!
      Am also looking into FUSE, and am interested in what webdav has to offer.
    • aurora wrote:

      I see, could you elaborate a bit more on why LAN protocols are so much slower than WAN protocls over the internet? Do LAN-based protocols operate on some different level of the OSI in how they handle networking?
      I waste already way too much of my time here so just a quick try: helios.de/viewart.html?id=787-en#Dir_Browsing (beware this is not only 'research' but of course also some advertising one vendor did for an own product that has been developed to solve exactly this issue).

      It's not only protocols that matter but 'default client behaviour'. Since you're a Mac user it's easy to explain: Open up any folder on a networked drive in Finder, run a sniffer in parallel and start to wonder why the display of a directory containing 10 pictures all with rather short names results not in less than 1KB as expected but in +10 MB traffic. Then think about how icons are displayed in this directory, look at the CPU utilization of processes called quicklookd and you get that part of 'openining a directory to see the contents' these days involves daemons lurking around in the background immediately opening file of different type to fetch additional information (worst case are image file formats not supporting embedded thumbnails).

      I would assume the same is happening on Windows too these days. And that's one of the many areas where those 'WAN accelerators' jump in: blocking all this stuff since understanding client behaviour and caching as much as they can to keep the WAN connection free (so usually they're a great solution to connect a remote office with some employees sitting around to a central server but won't help that much in your situation anyway)
    • Just install the plugin, set up the accounts for the users like anything else and make sure that you can access it from outside. Works just like outlook webmail if you will. You're just logging in through the web. So each user just uses their login account. Now there may be a bit more to it in omv with more than one user, so you may want to test that first because I'm not sure if they will all go into the same webdav folder or not if you use the individual home folder option which I think I saw somewhere. I don't use that so it may depend on how that's setup. I attached the image of the webdav in omv, it looks like you may be able to add all the different shares, but again, you will need to play with that as I don't have more than on and only set it up for me to test it.

      How is the performance of webdav? Looking through posts here, it looks
      like its not nearly as well supported as samba. I still need robust file
      permissions and user/pass authentication, which a post from 2015 says
      isn't supported yet?
      Could you link me to some further reading on how I can set up webdav
      with user/pass authentication and setting read/write permissions on a
      per-user and per-share basis?
      Images
      • webdav.PNG

        18.1 kB, 1,696×311, viewed 211 times

      The post was edited 1 time, last by Jaime ().

    • Jaime wrote:

      it looks like you may be able to add all the different shares, but again, you will need to play with that as I don't have more than on and only set it up for me to test it.
      No , you need to share top folder that contains your shares eg: share Photos and Photos contain 2017, 2016, 2015 folders

      performance: very good, please test it, I send *.avi files to my brother at max. speed of my internet conection.


      user/pass authentication : You need to log with you local credentials to acces your webdav files from outside; eg: you can access from your job to your home webdav shares , but need user/pass to access it, what user>>/pass, the user/pass that you create in OMV webGUI and have permision locally to acces files shared.
      OMV 4.1.11 x64 on a HP T510, 16GB CF as Boot Disk & 32GB SSD 2,5" disk for Data, 4 GB RAM, CPU VIA EDEN X2 U4200 is x64 at 1GHz

      Post: HPT510 SlimNAS ; HOWTO Install Pi-Hole ; HOWTO install MLDonkey ; HOHTO Install ZFS-Plugin ; OMV_OldGUI ; ShellinaBOX ;
      Dockers: MLDonkey ; PiHole ; weTTY
      Videos: @TechnoDadLife
    • Just thought I'd let everyone know, took the advice in this thread, and set up a VPN to my box.

      Using openVPN on my box worked perfectly, people thousands of miles away were pulling 100mb/s from the box back when it was set up. Unfortunately I can't get my drives mounted due to OMV timing out, but thats a different issue entirely. When everything was working, connections seemed quick and file transfer was fast as expected, although browsing folders with thousands of contents is noticeably slower, but nothing close to unusable.

      Thanks to everyone in this thread :)