OMV 5.x - Unstable SMB transfer speed

  • Hello OMV Community,


    Issue: I have an unstable transfer speed when writing a 22Go file to my NAS using SMB protocol. The transfer speed oscillates between 80Mbytes/s (640Mbps) and 112Mbytes/s (896Mbps) as you can see on this screenshot. The issue happens only when writing data to the NAS, not when reading data from the NAS. When reading I get constant 112Mbytes/s.


    Image/Screenshot: Unstable transfer speed from Laptop to NAS




    Laptop Setup:

    • Lenovo T460 - Fresh install of Windows 10 (version: 1909, build: 18363.535) with latest updates. Latest Lenovo website drivers (Bios + Hardware).
    • Processor: Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz, 2496 Mhz, 2 Core(s), 4 Logical Processor(s)
    • Ethernet adapter: Intel(R) Ethernet Connection I219-LM (Driver version: 12.18.9.8)
    • Installed Physical Memory (RAM): 8.00 GB
    • Storage (OS): SAMSUNG MZ7TY256HDHP-000L7 (238.5 GB)
    • Docking Station: Lenovo Thinkpad Pro Dock

    NAS Setup:

    • OS: OpenMediaVault 5.2.1-1 (Usul) - Fresh install, no modifications, all updates.
    • HP Proliant Microserver N54L (N54L)
    • Processor: AMD Turion(tm) II Neo N54L Dual-Core Processor (2.2 GHz)
    • Ethernet adapter: Broadcom Limited NetXtreme BCM5723 Gigabit Ethernet PCIe (rev 10)
    • Storage (OS): SSD 64Gb No brand
    • Storage (RAID): RAID 6 - 6 x 2To (7200rpm, Model: Dell MG03ACA200)
    • Installed Physical Memory (RAM):

      • MemTotal: 5963976 kB
      • MemFree: 5015820 kB
      • MemAvailable: 5261924 kB


    History:

    • I originally had OMV 4.x with the same issue, until I found a disk that had a red icon warning for relocated sectors, which I replaced with a another disk.
    • Once the array mounted, the transfer speed issue was still there, so I installed OMV 5 (not upgrade) but still got had the issue.
    • Then tried reinstalling windows 10 with all drivers/updates with no software tweaks/hacks/modifications, sadly the issue it still there.
    • I'm directly connected to the NAS using an CAT6 cable without any switch/router in between (although the issue was the same with a Fortiswitch 348B).
    • I tried using the SMB options from the "SMB Tunning" thread but to no avail.


    Question: What can I do to pinpoint the issue/bottleneck?


    Best regards,


    PS: Missing any information that could help?

  • Looks fairly normal to me, I suspect it's just buffering into RAM when it's fast and writing to disk when it's slower.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    I agree. The drive on the laptop has problems keeping up.


    Things that may help:


    Try to use a faster computer or a more efficient operating system on the receiving end. Turn off compression and antivirus. Replace the HDD with a SSD. Install more RAM.

  • Looks fairly normal to me, I suspect it's just buffering into RAM when it's fast and writing to disk when it's slower.

    I thought 6Go RAM would be enough. Is there a proper way to monitor this bottleneck, seeing when it starts to offload the content stored in RAM into the disks (raid)?


    I performed a dstat, does this help to confirm the issue?
    root@nas:~# dstat -c -D -g -l -m -n -p -r -s

    root@nas:~# dstat -c -d -g -l -m -n -p -r -s
    NAS_Dstat_monitor_during_transfer_2.txt


    EDIT: used wrong option, used -D instead of -d which did not display disk write/read information.



    I agree. The drive on the laptop has problems keeping up.


    Things that may help:


    Try to use a faster computer or a more efficient operating system on the receiving end. Turn off compression and antivirus. Replace the HDD with a SSD. Install more RAM.

    Isn't Gderf talking about the NAS's RAM?
    Anyway I tried with windows antivirus disabled, same issue.
    Haven't tried disabling compression, what is this?
    Also here is a screenshot of the a write/read test of the laptop's systems disk (it's an SSD):

    • Offizieller Beitrag

    This behavior is likely caused by buffers filling on the receiving end. It could be the sending end if the NAS is under very heavy load. But I doubt it would look like this.


    As data is received on the laptop it is first stored in fast RAM. At the same time another processes start writing the buffered data to slow disk.


    Typically the disk writes are faster than GbE network traffic. Unless you use stuff like antivirus scanning and compression. Or it is a lot of small files. Windows can use compression with NTFS. That may slow down writes.


    But the disk write processes are given a very low priority and buffers start filling up. The priority is set low because typically it is very small files that are written and typically the computer is idle every now and then. A form of opportunistic caching that usually is good.


    At a certain point the RAM buffers are so full that the operating system gives up buffering in RAM and the disk writing process gets full priority.


    This may rob the network receive process of processing power and the transfer speed is lowered. Possibly a lot.


    After a while the write buffers in RAM are small/empty again and the receive process speeds up again.


    And the cycle repeats.

    • Offizieller Beitrag

    Another issue might be the hardware makeup of the Raid 6, I use an N54L and there are 4 bays connected to the raid controller via the back plane, how are the other 2 drives connected.


    I'm not using a Raid I use mergerfs and snapraid and I don't suffer from this problem.

  • Thanks for the information. With smaller files it's fine, but not with some of my big files.
    I don't have any more RAM to test. :( Maybe one day I'll buy more.



    Another issue might be the hardware makeup of the Raid 6, I use an N54L and there are 4 bays connected to the raid controller via the back plane, how are the other 2 drives connected.


    I'm not using a Raid I use mergerfs and snapraid and I don't suffer from this problem.

    I have 4x2To disks in front bay connected to the backplane as you described.
    As for the tow other disks, one is connected to the motherboards' extra SATA port and the other is connected to the eSATA port on the outside.


    Do think that configuration could cause write speed issue or something?


    I don't know anything about Mergerfs and Snapraid yet. I'll look into it.


    EDIT: I'm using the BIOS mod in order to get all disks to work. Also my systems' disk (SSD) is connected to a SATA to USB adapter and then connected to a free USB port on motherboard.

    • Offizieller Beitrag

    Do think that configuration could cause write speed issue or something?

    Possibly, what you have set up makes sense but the bottleneck is probably related more to the hardware


    I'm using the BIOS mod in order to get all disks to work.

    So am I, 4 drives in the bay, one connected to the ODD port (CD/DVD sata port) one connected to the internal esata using an esata to sata cable.
    The 4 bays run the Mergerfs and SnapRaid, a 2.5" drive connected to the esata purely for docker configs and a large drive connected to the ODD port running rsync.


    It's very rare I copy from my server back to my workstation, but to give you some idea mine is the reverse of yours. Copying a file/s to the MergerFS pool it's possibly only writing to one drive, your Raid 6 will be slower due to the additional parity block it has to write.

  • What is the ACTUAL average speed of a single file large transfer something like 10GB or so?


    Because if your average speed is between 100-120MB/s then that's all you get unless you can manage to get SMB Multi-Path working. Which requires 2 ethernet connections on each machine that have separate IP addresses. I can't get it to work myself so I had to use 10G.

  • Hello,


    Please try to copy data to another destination and let us know if the problem persist… obs:
    Copy over network and directly and show us the result.
    How many RPM your HDD Laptop Have… if you can do speed test in different ways.
    Check duplex and your cables cat version and youe NIC because the transfer use the lowest media SPEED.

    The last time I saw eclipse I was programming :thumbup:

  • What is the ACTUAL average speed of a single file large transfer something like 10GB or so?


    Because if your average speed is between 100-120MB/s then that's all you get unless you can manage to get SMB Multi-Path working. Which requires 2 ethernet connections on each machine that have separate IP addresses. I can't get it to work myself so I had to use 10G.

    Hi, my average speed varies between 60MBytes/s and 112MBytes/s with a single 10Go file when transferring FROM laptop to NAS. It starts off fine but rapidly decreases and falls down to about 60MBytes/s for a second then goes back up to 112MBytes/s for a second as you can see in the screenshot. Yo-yo/roller coaster effect.





    I tried it with a single 1Go file, the results are a little better. The transfer speed holds up to 112MBytes/s for about 60% of the file but then starts to progressively decrease to about 90MBytes/s until the end. No yo-yo effect, but not full speed until the end either. See my second screenshot:





    I'm expecting to get constant 112MBytes average transfer speed on a 1Gig Ethernet link but I don't.


    As for the SMB multipath, I can't do that yet as I only have one ethernet port available on my NAS. In the future I could add a second port as I have a PCI slot available, but I my laptop only has one 1Gig ethernet port so that wouldn't work. Or would it?

  • Hello,


    Please try to copy data to another destination and let us know if the problem persist… obs:
    Copy over network and directly and show us the result.
    How many RPM your HDD Laptop Have… if you can do speed test in different ways.
    Check duplex and your cables cat version and youe NIC because the transfer use the lowest media SPEED.

    I will setup the test and come back to you with the results.


    • For my test, I'll setup OMV with a single 2To disk that is the same as the ones used for the raid 6. I'll use a seperate disk for the OMV system and one for the data.
    • I've specified the disk specification in my first post. They are 7200rpm, Dell MG03ACA200. 6 disks setup into an raid 6 array.
    • As for the link duplex/half-duplex, yes I'll check that to. Keep in mind, that when I perform my transfer tests, I connected the laptop and the NAS directly together without any switch/router in between.
  • I will setup the test and come back to you with the results.

    • For my test, I'll setup OMV with a single 2To disk that is the same as the ones used for the raid 6. I'll use a seperate disk for the OMV system and one for the data.
    • I've specified the disk specification in my first post. They are 7200rpm, Dell MG03ACA200. 6 disks setup into an raid 6 array.
    • As for the link duplex/half-duplex, yes I'll check that to. Keep in mind, that when I perform my transfer tests, I connected the laptop and the NAS directly together without any switch/router in between.

    Wow wait first try this before make any changes…. Backup file configuration or comment the original line and makes changes in new line.



    Type: vi /etc/network/interfaces


    up ethtool -s eth1 speed 100 duplex full autoneg off



    where eth1 is the name of ethernet adapter are you using on omv.

    The last time I saw eclipse I was programming :thumbup:

  • Wait? Wouldn't cut his transfer speed to 12MB/s?

  • Haven't changed anything yet, but are you sure about your command?
    What's the "up" in front of the rest of the command for? Is that a typo mistake? Or maybe you want to re-enable the interface because it will be down after commenting out the configuration in /etc/network/interfaces file?
    Also in my case, /etc/network/interfaces file doesn't have any configuration. Everything is done via the OMV web interface.


    EDIT:
    So I went ahead before waiting for your response and commented out anything in /etc/network/interfaces (there was nothing to comment out, static ip is set via web-gui), then executed this:


    I get an error when performing a transfer from laptop to NAS, see screenshot. The interesting part is the file is written to the NAS even though there is an error when transferring.


    Einmal editiert, zuletzt von baydry () aus folgendem Grund: Edit after performing test.


  • 1- the UP command is used to enable the interface.



    2- The 0x8007003B error is discused here and here


    3 - note that before make some changes i sugested to backup your config file it mean that you can restore your config if something go wrong


    4- after any config change you must restart related service in this case a network service.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!