How to install Squid Proxy to cache and speed up internet on your network

    Hello Open Media Vault community,


    There is a thread for suggestions on what should be made in to a plugin. In this thread I have mentioned that I think Squid Proxy is a good idea, and others agreed however asked for how I set it up so it can be converted in to a plugin.


    I would say I'm average skill level in linux, and absolute novice in Open Media Vault, so please excuse my steps below if they seem a bit choppy. I'm certainly happy for your feedback.


    Step 1: Gain access to your server
    SSH in to your Open Media Vault server. For me, my server has a local IP of 192.168.1.8, and I'm using Linux Mint on my notebook.
    Type this command in to a terminal window: ssh root@192.168.1.8



    Step 2: Download the Squid Proxy software
    Run the following commands;
    Update your repositories: apt-get update
    Install Squid: apt-get install squid3
    Check Squid version: squid3 -v
    Output should be long bu :!: t start with: Squid Cache: Version 3.1.20
    If it says version 2.x.x.x, then you installed the wrong package. remove squid, and install squid3 as per above.


    Step 3: Configure the Squid Proxy
    Open the configuration: sudo nano /etc/squid3/squid.conf
    Ctrl+W to search for something.
    Search for 3128 until you find: http_port 3128. Change that port to a port you can remember. I used 5555

    Code
    # Squid normally listens to port 3128
http_port 5555


    Find this line: acl CONNECT method CONNECT
    If your local area network uses 192.168.1.x (if not, change the numbers but leave the last as a zero as it's wildkey), add this below: acl mylan src 192.168.1.0/255.255.255.0
    Find this line: http_access allow localhost
    Add below: http_access allow mylan
    Find this line: http_access deny !Safe_ports
    This stops any non-standard ports from working, which is a pain in the ass. Replace it with: #http_access deny !Safe_ports
    Find this line: #cache_dir ufs /var/spool/squid3 100 16 256
    Remove the # at the start, then replace the '100' with how big you want your cache to be in megabytes. For example, mine is quite large at 20GB: cache_dir ufs /var/spool/squid3 20480 16 256


    You can tweak the other options. For example if you go down a few lines, you're able to change what files it will cache. By default it won't cache anything larger than 4mb. I have raised mine to 64mb. I also stopped mine from caching anything below 500kb.


    Ctrl+x and Y to save and exit.


    Step 4: Test the Squid Proxy
    Run this command to restart the server so we can read the configuration changes we made: sudo service squid3 restart
    Configure a your network to use your server as a proxy. In linux mint, I did this from the network settings:

    Then test your web browser by navigating to a web site. Make sure the web site does not use HTTPS. Squid Proxy cannot work with SSL as it can't intercept an encrypted connection.
    If it works and the web site loads, yay! If it doesn't, post below so we can troubleshoot.


    The only real way of knowing if the proxy is caching, is to read the logs. Hit means it was in your cache so it was not downloaded. Miss means it was not and Get means it was not however it will be next time as it meets the criteria of caching and has now been saved.


    You can configure your squid proxy to pre-fetch web sites for even faster browsing, to block pop-ups and adverts, to scan for viruses on web sites etc.


    Another handy thing is to generate reports:
    http://www.tecmint.com/sarg-sq…andwidth-monitoring-tool/


    Let me know what you think by giving a thumbs up on the bottom right corner if this is good.

    Hi @luckyjay,


    Just offering my two cents here. :) Things like squid proxy server should be left as a separate item from OMV. The reason I say this is because setting up proxies or reverse proxies should reside in a vm, A firewall app such as pfSense, or a docker container possibly. I don't think it is beneficial to have this as a plugin to OMV. I could be wrong though. I am just speaking from experience. If anyone needs to implement a proxy on their home network, I strongly advise you do it outside of OMV and run it either as a VM, Docker container, or part of a firewall solution such as pfSense. You can easily block yourself out of OMV without knowing the right firewall rules to implement. Again just my thoughts about the subject. I am no way discouraging the thought of a plugin but I just think it's not a good fit. Things could go wrong really fast for a new user in my opinion.

    ShadowZero -- OMV Fan since 0.3

    Hey,


    I don't see how it's possible to lock yourself out of OMV doing this. This isn't a firewall at all, it's an optional proxy. Your devices will work if they don't use this proxy, its an optional proxy. Only devices configured to use it will do so.


    I appreciate your input but am not sure I understand how this could block someone out of OMV?

    I was thinking about if users turn on transparent proxy on port 80. But your instructions don't include that so it should be fine. Again sorry for the confusion. I was thinking about my setup with squid which I use transparent proxy settings. :thumbup:

    ShadowZero -- OMV Fan since 0.3

    Once again I feel like I am confusing your idea for the plugin. ;) Sorry about that. Transparent proxy set to port 80 would filter all traffic through the proxy without setting the port to 3128. Using this method you do not need to add the proxy information to chrome, firefox, etc.. It's just another method to use Squid. I use it this way because I can set filters for my kids and not have to set any proxy information on any device they use. Plus it keeps any guests to my home from visiting websites I think they should not have access to.

    ShadowZero -- OMV Fan since 0.3

    Oh that sounds quite convenient actually, must be how corporate and public networks configure there's. I've never tried that before, but yeah you're right, you'd have to configure OMV on an alternative port first

    Uh, this looks like a spaghetti of assumptions.


    As a basic concept: "proxy speeds up the Internet" does not mean it will always make internet access faster. It can only make some pieces download faster (from a local cache than from the original site), as long as they have been accessed before and kept in the cache. But your browser already does this anyway, even without a proxy. So the actual savings of traffic are visible in case of large groups of people accessing pretty much the same internet content, and when you tweak the proxy's content retention period to be longer than the browser's cache.


    To explain where the proxy fits into the picture, consider that you have 3 ways of connecting to the Internet:

    • You want your workstations to connect "directly" to the Internet. For this, you get a router to do packet routing between your LAN and any other destination, and configure the workstations with the router as "default gateway". The router can prioritize various types of traffic and allow/disallow some connections according to a whitelist/blacklist, but other than that you're not doing any sort of connection logging, content inspection, etc.
    • You want to fully control what your users are accessing, and for this you do not define a default gateway. Only specific machines on your LAN are able to access both your LAN and the Internet - for instance, a web proxy server. For this, you define a proxy server and your clients are required to explicitly define it in browser's settings, otherwise they don't have Internet access. The proxy server works at application level in the OSI model, so this allows you to do some fancy stuff which a router would not be able to do, for instance: do content caching, whitelist/blacklist URLs based on wildcards, have full logging of all traffic for audit purposes, etc. But, clients must be expressly configured to use the proxy.
    • You want the advantages of having a proxy, but you don't want your users to manually configure browser, or to even know that the traffic is passing through a proxy and is being logged. Or maybe you need the clients to first authenticate themselves in a browser before getting Internet access (like some corporations, hotels, etc.) In this case, a transparent proxy will be used. From the client perspective the configuration is similar to a router (the DHCP server gives you an IP, a DNS server and a default gateway), however that gateway actually is not just doing packet routing, but also traffic inspection and classification. It then decides whether the traffic is HTTP(S) or not, and what to do with it based on some rules. And for HTTP it can leverage a caching mechanism in a completely transparent way to the end user.


    Not to confuse this with a reverse proxy - this sits on the server side. For instance, I may be running a website on a web host somewhere, and the database is getting hammered by requests for every single pageview. One of the options to limit this is to route all traffic to the website through a reverse proxy, so that the proxy can cache the pretty much static pages/content and deliver them to the visitors, and avoid a load of database hits; only the unique requests (specific to one user or another) would pass through the reverse proxy to the web server to be generated.


    So, theory aside :) If you want to have a proxy on the LAN, you need to figure out if you're in the scenarios 2 or 3 above. You can even set up a proxy server on a networked machine just for caching (allow direct traffic through the router), and point your other machines' browsers to it as a proxy for HTTP. For a transparent proxy, things are trickier.


    Some other things to keep in mind:the proxy machine needs to be seriously beefed up with RAM and spare disk space for cache;

    • the traffic savings become relevant for large organizations, and pretty low for a home use scenario - see the basic concept at the beginning;
    • a lot of the web content these days is dynamic, so the savings will vary a lot depending on the internet use patterns of your people;
    • in order to be effective, the proxy needs to find a very fine balance and sensible configuration/management, so that you don't run into various issues; for instance, you may force items to expire after a looooong time so that they are delivered from cache, but this may prevent visitors from seeing new content; on the other hand, you may leave the expiration time too loose, in which case the proxy rarely serves content from cache and just forwards the request to the Internet most of the time, which is not really effective, or not significantly more effective than the browser's own caching mechanism.

    Not tried the firewall in OMV, but I'm sure I read you can just filter addresses with iptables.


    iptables -A OUTPUT -p tcp -d http://www.badplace.com -j DROP


    Actually I have started with a minimal debian install on a raspberry pi forwarding ipv4 traffic from an AP USB WLAN to have a guest wifi network. I will try with iptables and see how it goes. pi's are neat little things but a pi1b really isn't suited to squid3(even with cache-dir disabled)

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login