Very slow network transfer speeds

  • Hello!


    I've been using OMV2 for a while. My System specs are:


    AsRock AM1B-ITX
    AMD Athlon 5350 @2.05Ghz
    8GB RAM
    2x Seagate ST4000LM016 (2x 4TB, RAID0 with ZFS)
    1x Crucial M4 128GB mSATA
    OpenMediaVault 2 (Backports Kernel)


    The problem is, the transfer speeds TO the NAS are very slow, it doesn't matter which protocol is used. NFS is slightly faster than Samba and FTP, but 20MiB/s are just too slow. When reading from a Samba share I get about 110MiB/s, so almost everything the gigabit network can passthrough. When I write something to the share, I got about 6MiB/s.
    I tried already several Samba tuning options. The well known socket options, as well as sendfile, aio read/write behind and some others. Nothing helped.
    The strange thing is, sometimes I get high transfer speeds of about 80MiB/s on large files, but if I then copy another large file it's slow again. The speed isn't stable and I have no clue how to improve this. Any advices?
    The network speed is stable, tested it with iperf very often, it's always about 940MBit/s. My clients running all with Ubuntu.
    This isn't a problem about the speed of the ZFS pool, tested it already with transfering a file from an external hdd directly attached to the NAS. I got consistent speeds of up to 120MiB/s.

    • Offizieller Beitrag

    Have you tried max and min protocol parameters?

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    I missed that you have slow write speeds with multiple protocols. I know iperf gives good results but we have seen this plenty of times with realtek NICs. If you don't want to move to OMV 3.x, there is a thread that has some 4.x kernels that might improve things.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Do you mean this one?

    Yep. If it doesn't fix it, I generally recommend getting an Intel NIC or compiling the realtek driver yourself (yuck).

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Same problem here, W=30MiB/s, R=100MiB/s
    Writing starts at 80MiB/s and quickly breaks down :(


    OS Win10, Nic is Intel 82579LM
    NIC in the OMV:

    Code
    tg3 0000:04:00.0 eth0: Tigon3 [partno (BCM57780) rev 57780001] (PCI Express)



    Any links to the max/min protocol? Haven´t tried this yet...


    thx

    --
    Get a Rose Tattoo...


    HP t5740 with Expansion and USB3, Inateck Case w/ 3TB WD-Green
    OMV 5.5.23-1 Usul i386|4.19.0-9-686-pae

    2 Mal editiert, zuletzt von Dropkick Murphy ()

  • Well, I'm not sure, if an Intel NIC helps anyway...
    My other NAS, running with Ubuntu 14.04 Server, is using a HP NC364T. The write speeds was as unstable as the speeds on the small NAS with OMV2. Even in bonding mode round-robin (4 NICs), the speed was often down at 11MiB/s though my directly connected desktop pc had a 364T, too.
    After upgrading to a newer kernel (4.x), because the old 3.x kernels were no longer supported, the speed problems are just gone. But I'm not sure it is related to the driver module, because I compiled the module by myself before and it didn't help either. It seems to be something else kernel related.


    Today I tested kernel 4.5 with my OMV2 NAS, it seems to work. I transferred some files with sizes from 8GiB to 20GiB and got at least 60MiB/s. Compared to before, this looks great :)



    @Dropkick Murphy
    just write the line "max protocol = SMB2" (without "" of course) to your smb.conf and test if it helps.


    Some links:


    https://www.arm-blog.com/samba…r-better-transfer-speeds/


    http://lug.krems.cc/docu/samba/appb_02.html



    The second one contains a list of recommended samba settings at the bottom of the site.

  • Thanks so far...
    smb2 did a little better, the rest of the tuning tips didn´t work (write rate even worser).
    I did install the latest nic driver from Intel and flashed my HP-Switch to the latest FW.
    Now, I´m here:


    Thats even better...


    Thx

    --
    Get a Rose Tattoo...


    HP t5740 with Expansion and USB3, Inateck Case w/ 3TB WD-Green
    OMV 5.5.23-1 Usul i386|4.19.0-9-686-pae

  • Ok, very strange...
    After it did work this morning, now, after a reboot of my NAS, I'm back to low speed again...
    I noticed something strange, if I take a look on the RX/TX speed bmon shows (on the NAS), the RX speed matched with the TX speed on my client. But in addition, the TX speed of the NAS is very high, but why, it is a write from the client to the NAS, so what does the NAS transmit? There is no read transfer running.
    The transmission stops immediately after canceling the file transfer, so there is any kind of relation. After some testing, I can say, this problem is related to Samba. NFS do have some slow downs while transfer, but the overall speed is about 60MiB/s. Any ideas?


    [spoiler}[/spoiler]

    • Offizieller Beitrag

    NAS is very high, but why, it is a write from the client to the NAS, so what does the NAS transmit? There is no read transfer running.

    Samba is a higher overhead protocol that talks both ways even if it isn't reading.


    Any ideas?

    I wish I had a system that had this problem so I could try more things to help. I would keep trying more tuning parameters and look for other things maybe in a samba forum or try their IRC?

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    But the client receives nothing, so where is this overhead going to? If i open bmon on the client, it receives just a few kiB/s.

    I don't know a lot about samba protocol. I just know it is a heavy protocol (communications-wise) compared to nfs and ftp. You would have to use something like tcpdump to figure it out.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Ok, now I'm pretty sure I tracked it down.
    In the first place, the problem was a bad Realtek driver module, I guess. This was solved by using a new kernel. Due to some bad Samba server configuration, I had that strange behaviour described above. Sometimes I got 110MiB/s on writes, but mostly I got 5MiB/s.
    What I did now was to disable any tuning option in the smb.conf, including the socket options. Now I got at least 70MiB/s on writing large files (tested with 10GiB files or even bigger). Next step was to set only two options, "aio write behind = true" and "write raw = yes". Now I get an average of 110MiB/s on large file writes and at least 55MiB/s on small file writes. It's stable, I tested it with several files with an overall size of about 200GiB. I even did some reboots to ensure, this doesn't change anything.


    Well, I guess I'm fine with it... :D


    @ryecoaaron
    Thanks a lot for your help!

    • Offizieller Beitrag

    Now I get an average of 110MiB/s on large file writes and at least 55MiB/s on small file writes.

    Those are good numbers. Glad it is working :)

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!


  • What I did now was to disable any tuning option in the smb.conf, including the socket options.
    [...]
    Next step was to set only two options, "aio write behind = true" and "write raw = yes".

    Could you please explain it more particularly for me? What exactly options should I disable? And what about setting that two options you mentioned - just adding them to smb.conf into "Global Settings"?

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!