Fixed upload rate for ALL users or the ProFTPd service itself

  • Hello OMV-Community!


    I'm getting crazy... Searching for a possibility to limit the FTP uploadspeed to a fixed amount, because if multiple users or one user with multiple connections is/are downloading, they reach my max. upload speed which is about 1280kB/s in total. Then my ping raises like crazy and I cant play properly anymore. I tried multiple things, for example: the linux tool trickle and 2 different QoS. Trickle won't affect proftpd, I dont know why - it just wont limit it. DD-WRT QoS also did not work for the wired server. Only for the wireless clients (I dont know if its a bug or wrong configuration of the router itself, but I just can't see wired clients in the interface of DD-WRT (we use it as Access Point, main router is a Fritzbox 7390 with deavtivated WiFi. DHCP in DD-WRT is turned off or used as forwarder).


    OMV-Server---Gigabit LAN--->DD-WRT---Gigabit LAN---> Fritzbox.


    Fritzbox QoS also doesn't work at all, not even with own, newly added rules with the ports which some games use (CS:GO for example)


    So, this is why I want to limit the ProFTPd transferspeed in overall, not per "TransferRate", as this only limits the speed per connection.
    http://www.proftpd.org/docs/di…fig_ref_TransferRate.html


    The OMV-WebUI does use the same command.
    If I limit the connections to 1, the users can't browse anymore when downloading (when they are using a normal browser for downloading, firefox for example), if they use a FTP client like Filezilla, they can't download anymore, because the server always uses permanently 1 connection for the directories. Also if multiple users download at the same time, they always would reach the max. uploadrate, no matter how much connections per user are set.


    subzero79 tried to help me over IRC, but its too hard for me as a linux newbie, to compile and install ProFTPd with the trafficshaper-module.
    Is there a way to request this feature to be implemented in one of the next versions? Who must I ask for this?
    The module: http://www.proftpd.org/docs/contrib/mod_shaper.html


    Thank you in advance!

    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black

    Einmal editiert, zuletzt von Nyctophilia ()

  • You should create a feature request at http://bugtracker.openmediavault.org/


    And all your attempts to limit the transfer rates didn't work? Well, I made the experience that proftp running as a plugin on OMV is a litte fussy if you want to change the standard settings.
    For example, if you enter a command like DefaultRoot in the extra options of the plugin, this command is saved to the proftpd.conf file, but the server doesn't use it. Why? After checking the proftpd.conf I saw that the DefaultRoot direcctive appears twice.... the first one is the standard setting (For me this seems to be hardcoded) "DefaultRoot /srv/ftp" and some lines below my command "DefaultRoot ~", but the second one seems to be ignored.
    The only way to tell proftpd that it should use the tilde for leading users to their home dirs is to enter a line in the OMV default config file at /etc/default/openmediavault using the correct syntax and then it works.


    To make a long story short, did you try to use the correct settings for limiting the transfer rates in the default file? I would enter a line like OMV_PROFTPD_TRANSFERRATE="RETR 20.0", run "omv-mkconfig proftpd" and "service proftpd restart" from the CLI.
    If you then enter some limitations for the number of connections, say max. 5 clients and max. 2 connections per client this should end up in a max. transfer rate of 200kb overall. Give it a shot and tell if that works.

    Homebox: Bitfenix Prodigy Case, ASUS E45M1-I DELUXE ITX, 8GB RAM, 5x 4TB HGST Raid-5 Data, 1x 320GB 2,5" WD Bootdrive via eSATA from the backside
    Companybox 1: Standard Midi-Tower, Intel S3420 MoBo, Xeon 3450 CPU, 16GB RAM, 5x 2TB Seagate Data, 1x 80GB Samsung Bootdrive - testing for iSCSI to ESXi-Hosts
    Companybox 2: 19" Rackservercase 4HE, Intel S975XBX2 MoBo, C2D@2200MHz, 8GB RAM, HP P212 Raidcontroller, 4x 1TB WD Raid-0 Data, 80GB Samsung Bootdrive, Intel 1000Pro DualPort (Bonded in a VLAN) - Temp-NFS-storage for ESXi-Hosts

    • Offizieller Beitrag

    You need the dev package from proftpd


    apt-get install proftpd-dev


    Noe we grab the source from debian same version for wheezy


    https://packages.debian.org/wheezy/proftpd-basic
    http://ftp.de.debian.org/debia…-dfsg_1.3.4a-5+deb7u2.dsc


    git clone git://git.debian.org/pkg-proftpd/proftpd-dfsg.git


    cd proftpd-dfsg/contrib


    prxs -c -i -d mod_shaper.c


    That's it your finish. The shaper goes in this simple way in extra options


    Code
    ShaperLog /var/log/proftpd/proftpd.shaper
    ShaperTable /etc/proftpd/shaper.tab 
    ShaperEngine on
    ShaperAll downrate 15 uprate 15


    For advanced options read the proftpd doc website. Is sleeping material but....

  • You should create a feature request at http://bugtracker.openmediavault.org/


    And all your attempts to limit the transfer rates didn't work? Well, I made the experience that proftp running as a plugin on OMV is a litte fussy if you want to change the standard settings.
    For example, if you enter a command like DefaultRoot in the extra options of the plugin, this command is saved to the proftpd.conf file, but the server doesn't use it. Why? After checking the proftpd.conf I saw that the DefaultRoot direcctive appears twice.... the first one is the standard setting (For me this seems to be hardcoded) "DefaultRoot /srv/ftp" and some lines below my command "DefaultRoot ~", but the second one seems to be ignored.
    The only way to tell proftpd that it should use the tilde for leading users to their home dirs is to enter a line in the OMV default config file at /etc/default/openmediavault using the correct syntax and then it works.


    To make a long story short, did you try to use the correct settings for limiting the transfer rates in the default file? I would enter a line like OMV_PROFTPD_TRANSFERRATE="RETR 20.0", run "omv-mkconfig proftpd" and "service proftpd restart" from the CLI.
    If you then enter some limitations for the number of connections, say max. 5 clients and max. 2 connections per client this should end up in a max. transfer rate of 200kb overall. Give it a shot and tell if that works.



    I used the commands with the "Extra Options" field. Writing it directly into the ProFTPd config did not help at all, because it reseted when restarting the server or reapplying the config per WebUI.


    The thing is - I want my users to able to download with as much speed per file as possible - no weird combining with connections to get a total rate :P They should be able to download one file with max. bandwidth, indepentend from how much users and connectons are used. That's why the shaper_module is probably there. But I'm too stupid for Linux yet. Luckily subzero79 wrote me a step by step tutorial. I'm gonna try it now :)


    Im also gonna add the request to OMV-Bugtracker - maybe there is interest for it :)

    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black


  • apt-get install proftpd-dev worked so far - no problem - but I dont know how to grab the source? What exactly should I do with these 2 links? Downloaded the source now with wget.
    https://packages.debian.org/wheezy/proftpd-basic
    http://ftp.de.debian.org/debia…-dfsg_1.3.4a-5+deb7u2.dsc


    how do i get the git:// link? Or what else am I supposed to do with it.


    Sorry, im really a CLI noob and I only know a few basic linux/unix commands :/
    Using Linux for about 3 weeks or so.


    Thank you for helping me

    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black

    Einmal editiert, zuletzt von Nyctophilia ()

    • Offizieller Beitrag

    You can request at bugtracker but remember that proftpd is a package distributed by official debian repos. This feature does not come with the package by default. So a feature to enable this is gonna be useless if the user doesn't build or add this module by his own.
    And is for this reason that we have extra options in global and shares.


    @datadigger The other day I pointed you to use the ~ in DefaultRoot is still not working for you in the environmental variable?


    I tested this the other day and it worked perfectly

  • Okay, I did it exactly as you wrote it down, but still it doesnt shape it to the wanted amount :/


    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black


  • After compiling it also said:



    But I can't add it, because it always resets the config when reapplying/restarting the server - if i put it into the "Extra Options" tab, there is an error. If try to fix this error by commenting #LoadModule mod_tls_memcache.c, it still gets me the same error.



    (I understood why there is this error now, because it's already loaded.)

    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black

    2 Mal editiert, zuletzt von Nyctophilia ()

    • Offizieller Beitrag

    Is already loaded as you can check also in /etc/proftpd/modules.conf


    You have your module now in proftpd is time for you to understand how to configure it properly. The example I posted was very simple. This module maybe it works maybe it doesn't, Is hard to find examples of proftpd since is very old. But the doc website indicates many possible configurations
    Check also the log at /var/log/proftpd/proftpd.shape to see whats going on


    Other thing would be now to use traffic shaping rules by port or something. Traffic shaping rules is not something that and I am familiar with.


    The utility trickle can limit bandwidth to commands but not for PID unfortunately

  • I can't use http://www.proftpd.org/docs/co…d_shaper.html#shaper_info - Is this needed for the usage/configuration?


    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black

  • Found a perfectly working solution.
    Followed this tutorial: http://www.cyberciti.biz/faq/l…ng-ftp-server-port-21-22/


    If you are using normal ftp without passive ports, then you should only set rules for port 20 and 21 (This is the reason why the shaping per iptables and tc never worked for me... I did not know that ACTUALLY FTP PORT 20 is used for the filetransfer. If some guys on IRC.freenode.net ##networking would not have told me that, I still wouldn't know that... Right now, I am using a passive range and it works fine.
    Limited to 1000kB/s outgoing, automatic/dynamic bandwidth dividing and no ping rises at all. Finally!


    Adapted these 2 lines and changed them to my needs:

    Code
    /sbin/tc class add dev eth0 parent 11: classid 11:1 htb rate 1000kbps ceil 1000kbps quantum 2048
    /sbin/tc class add dev eth0 parent 11:1 classid 11:101 htb rate 500kbps ceil 1000kbps prio 0 quantum 2048


    To make everything permanent and working after a reboot, people in ##networking suggested me to do:

    Code
    /sbin/iptables-save > /etc/firewall.conf; echo "/sbin/iptables-restore < /etc/firewall.conf" >> /etc/rc.local; echo "ENTER YOUR TC COMMANDS HERE" >> /etc/rc.local


    Then open up /etc/rc.local with an editor and it should look like that (exit 0 must be at the very bottom, atleast that's what people told me):


    This works fine for me, but If you guys have some better and more efficient solution, just post it!


    Some guy on IRC.freenode.net #proftpd brought me to the idea to shape it per iptables and tc, as it is much more efficient. I just love linux and the community. Getting more and more used to it and I'm having some serious fun by learning and actually doing some useful stuff with it! Thread solved!

    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black

  • Yeah, it is kinda useless. The shaping was awful when I got it to work. The iptable rules are the same as in the guide, I'm just using other ports for the passive range.

    OpenMediaVault 1.12 on Debian Wheezy 7.8 64bit | 3.16 Backport Kernel | OMV-Extras.org 1.10
    ASRock Q1900-ITX | 1x4GB Corsair Vengeance SODIMM DDR3 | 1xWD Red 3TB | 1xApacer 1,8" SATA1 SSD 32GB as system drive | be quiet! Pure Power L8 350W | SilverStone Milo ML04 Black

    2 Mal editiert, zuletzt von Nyctophilia ()

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!