SMB performance INCREASE with two transfer

Hi guys,

let's take a look to transfer speed from my NAS to my Computer:

Ok, more or less 5 MBytes/s. Quite disappointing.

But... try to start another simultaneous transfer, from the same NAS, same disk, same folder, to the same computer, same disk, same folder:

Oh my god!... The first file now is running 6 time faster!

The total transfer speed now is 60MBytes/s, and it is able to saturate the bandwidth of my local disk!

Let's try to pause the second file:

Good! The behaviour is repetible: now, with the second transfer paused, the first one dropped down to its initial speed of 5Mb/s.

It's magic! It's samba .

Going into detail, and adding some argumentation...

I noticed this behaviour some months ago, when I installed OMV6 on a machine where OMV5 was present and worked for years. I thought a lot before writing this thread, because SMB performance is a problem widely documented on Internet.

I found a lot of guides about how to improve SMB performance editing some configuration: example Link 1, Link 2, Link 3.

What makes me confused is the fact that in several posts/threads/websites, I found sentences like this:

Quote

(the next sentence is a modified version of the most clear advice that professional users generally give to people that complains about low SMB performance)

It does not make sense to post different smb settings as these settings are serving a purpose. So they might make sense or not depending on the use case.

If you want to check whether or not a setting makes sense for your situation, it might be best to consult the documentation

Link to official SMB documentation - If printed this document is more or less 190 pages long.

I understand this advice. And I am nerdly enough to read the destination document in the most interesting parts for me.

Bad for me that I can't know if, for my Windows, is better max xmit = 65535 or max xmit = 61477 or max xmit = 25981. And, I know, this is only a provocative example useful to explain that I cannot run 65535 benchmarks in order to find which value I should set to a single SMB parameter.

What I am asking to mystelf, since I would like to understand and learn, is:

• why in my scenario is SMB joking with speed?
  I think to be a very normal user - a user which represents the mean: I own a normal PC, with Windows, that is the most used OS in the world. I installed OMV on a old PC that is powerful enough to fill completely my needs. I own also other devices like tablet and smartphone with Android that is the most used smartphone OS in the world.
  So, why should I change the default configuratio of Samba? Is the Samba OMV's configuration optimized for other scenarios? Please correct me as well, if you think this is not true.
• if the configuration on SMB has to be changed, how can I know which knob should I touch?
  The configuration of SMB is very detailed and can be adjusted in hundreds of ways.
  It is not scientifically acceptable that somebody says me "make some tests and try to find your best configuration", simply because the combinations are more than the time I have left to live. Again, correct me if my reasoning is wrong
• Probably the best configuration for Windows 11 is different from the best configuration for Android, and it is different from the best configuration for a Firestick TV.
  So, should I give up and choose a medium configuration? Cannot SMB itself negotiate the best configuration between two devices?
• Does exists a software that analyze the ethernet traffic and reccomend me an optimal configuration?
• In other words, does exists a scientific way to tune SMB?

HW configuration

The screenshots and the tests have been taken with this hardware configuration:

NAS: OMV6, latest upgrade, Gigabit connection, no SMB "modding" - only standard configuration. Intel Q6700 @ 2.66GHz. 4 GB Ram, 26%used. No Docker Containers running during the tests.

PC: Core i7/9700K, destination disk WD15EARS without load during the test, operating system Windows 11 latest version. 16 GB Ram. No apps was running during the test.

Thank you to share with me knowledge I don't have.

Thank you for your answer ryecoaaron.
Every comment to my initial post is welcome, because I'm here to learn.

First I would like to say that I have full faith in OMV: I have been using it since version 3 and any problems I have ever encountered with this software have never caused me any resentment, as I know very well that it is written by people who did it in completely free way. So I have only to thank both the programmers of OMV and this forum itself.

Quote from ryecoaaron

There are so many factors that play into samba speed and many of them are the client and the network. Some users are using slow wifi and others are using 10GBe. Some people have very invasive virus scanners that can significantly slow down transfers.

Yes, I understand. This is why I tried to be as detailed as possible. Seems to me that my test is demostrating that disks and hardware parts are not the bottleneck, otherwise I would not be able to reach speed of 60MB/s (limited in my tests by PC local disk). The same is true for the Antivirus (that is the default one provided with W11).

So I start my argument assuming the problem is on the software.

Am I sure it’s SMB’s fault? No, absolutely no.

Is it reasonable to think it’s SMB’s fault? From my point of view, given the amount of complaints that run in the www, yes.

This is why I asked if there is a scientific way to tune SMB parameters.

Quote from ryecoaaron

You mention that there are 190 pages of samba docs and that you shouldn't be the one to go through that.

I say also that I read this document in the most interesting parts for me: as a computer enthusiast, I have no problem studying a new topic. The problem is that even if I learn how to configure a samba server, I still don’t know which configuration to assign to it because I don’t know what the client wants.

Quote from ryecoaaron

There is only one author of OMV and he doesn't use samba last I checked. Even I don't use samba since I don't use Windows.

It is not clear to me what you want to say with these sentences. Are you saying that if you && the author of OMV are not using SMB, its default configuration is abandoned to itself? Or are you saying that "a tipical an OMV user" doesn't use SMB because probably its main OS is Linux?

Quote from ryecoaaron

The good news is that this is not an OMV issue.

No it isn't. Unless the default configuration can be the issue.

Quote from ryecoaaron

So, you can look at everything on the internet for tips not just OMV related. I will mention that samba itself has to disable older protocols that are full of security holes. These older protocols usually have less overhead and can yield higher speeds.

Again, I have nothing against OMV. But the fact that I am able to reach high speeds makes me hope that it is a question of adjusting some parameters, but I would like to find them in a scientific rather than experimental way.

Quote from ryecoaaron

I know you are looking for a short magic answer to the problem but there isn't one.

Thank you to inform me about this. Sometimes, with Linux, magic happens. There are people out there who have big brains. It is not uncommon for these people to write free software, and go deep into a problem.

Thank you again for your help and the time you spent for me, ryecoaaron.

Quote from donh

Can you try transferring between two windows computers on your network?

Unfortunately no. I have no a second windows computer with gigabit (1Gb) connection.
But I have another PC with Linux Mint installed, and this is the SMB throughput between them:

This screenshot rapresents the download of a 4GB file from Linux Mint to Windows 11.

Some time ago I also reached these speeds with OMV.

Quote from geaves

TBH we are assuming that the network connectivity is 100%, check the connection between the server and client and vice versa is sound by using iperf

Thank you very much geaves for your hint. This is the result:

192.168.0.5 is the IP of my OMV NAS.

I'm investigating, and I'll keep you informed if I find something interesting.

Today I made some test:

1) I swapped my Linux Mint computer with my NAS and repeated the speed tests made above. This test is useful in order to exclude 100% every hardware related problem on the ethernet/hardware side (eg cables, switch connectors, power source, electric noise, cables lenght, etc..).
With unit swapped the speeds don't change: Linux Mint near 100MB/s, OMV near 4/5 MB/s. with one transfer in progress.

2) I tried to upload a file from W11 to NAS. The speed is above 99MB/s during all the transfer.

3) I tried to configure SMB with some "extra stuff" linked on my first post. Nothing change. Retuned to original default configuration after this test.

So the sure things are:

1) I need to consider good the hardware part.

2) The issue is present only when one transfer is in progress. Does not matter if the download device is W11, Android or Linux.

3) The issue is present only in "download" mode. Upload runs fast.

4) The issue is present only with SMB protocol. Download and upload with NFS runs very fast @95-100MB/s.

Quote from geaves

Interesting, have you tried this between Linux Mint and OMV

Yes I tried. The speed is slower than Windows 11. A transfer from NAS to Mint runs @ 1.6MB/s.

Quote from geaves

do you have IPv6 enabled in your network settings on W11

Since I know nothing about IPV6, and I don't know if it can be "dangerous" to me, I disabled it in OMV and also in my router.
So both OMV and Windows 11 have not an IPV6.

Today, for test, I disconnected all my original disks from the server.

I reinstalled OMV from scratch into an SSD., and I have configured it with a SATA disk spinning at 10.000rpm, so I can run some benchmarks and experiments without the fear to break my original configuration.

After the inital, very basic, setup the speed is very low, as my original OMV system:

SMB: read 5MB/s, write 99MB/s.

NFS: read and write at 95MB/s.

I made also some modification in BIOS, disabling some "ECO functionalities", but nothing changes.

I installed an external USB 2.0 to Ethernet adapter.

With this adapter the SMB read speed grows up to 11MB/s, but this test is pretty insufficient to determine if this can be the problem, because:

1) USB2.0 have a very limited badwidth. (There are not USB3 connectors in this system)

2) also with original NIC I can reach 60MB/s of download speed, if i start TWO SMB download simultaneously.

I found another user with a similar problem: here . Also in his experience, there is a low performance ONLY in SMB and ONLY in ONE direction.

Quote from geaves

So NFS works as expected for Read and Write from the server -> W11, Linux Mint, Android

Not excaclty. I tested NFS only with Linux Mint.

Quote from geaves

SMB works as expected for Write but degrades for Read from the server -> W11, Linux Mint, Android

Yes. Tested with all OS you mentioned. Remember that speeds improve drammatically every time I start a second download.

Quote from geaves

Even I'm having trouble understanding that, if there was the same issue on NFS as on SMB then we would be looking at a hardware problem, some where

I've tested NFS on my W10 and I would use it except for the fact that you have 'map' a drive letter to a share, something I've never done, my shares are viewable in File Manager -> Network

Are your SMB shares mapped as drive letters

Re IPv6, for local networking it's not necessary at some point in the future when more of the internet moves to IPv6 then the change will be inevitable + if you do things like nslookup IPv6 always seems to be first in the queue.

Yes, also for me it's a trouble to understand. I think the problem cannot be in the hardware because of the tests I done. But if you have other tests to propose, feel free to tell me which.

No network mapped on drive letters for me. This is not the problem, because Linux Mint cannot "map letters".

I Know that IPv6 is very important. Sooner or later I will study that too. As a hobbyist, I haven't done it yet.

Quote from donh

What nic hardware are you using? lspci

#lspci
02:00.0 Ethernet controller: Qualcomm Atheros AR8121/AR8113/AR8114 Gigabit or Fast Ethernet (rev b0)


#lspci -kn
02:00.0 0200: 1969:1026 (rev b0)
        Subsystem: 1043:8226
        Kernel driver in use: ATL1E
        Kernel modules: atl1e

#lshw
           *-network
                description: Ethernet interface
                product: AR8121/AR8113/AR8114 Gigabit or Fast Ethernet
                vendor: Qualcomm Atheros
                physical id: 0
                bus info: pci@0000:02:00.0
                logical name: enp2s0
                version: b0
                serial: [MAC address removed by TheFax]
                size: 1Gbit/s
                capacity: 1Gbit/s
                width: 64 bits
                clock: 33MHz
                capabilities: pm msi pciexpress bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
                configuration: autonegotiation=on broadcast=yes driver=ATL1E driverversion=5.16.0-0.bpo.4-amd64 duplex=full firmware=L1e ip=192.168.0.169 latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s
                resources: irq:17 memory:febc0000-febfffff ioport:ec00(size=128)

Results have been cleaned up in order to remove all infos not network-related.

Finally, after several tests, I bought a new Network Card, based on Intel 82756 as andrzejls suggestion.

With this intel card the speed is good in both direction, download and upload, and both via SMB and NFS.

In the mean time, when I was waiting for the new NIC, I studied the problem and try to do some kind of troubleshooting using different methods (wireshark, tcpdump, changing SMB parameters, using different versions of Linux Kernel, loading the atl1e driver with different parameters), but I haven't been able to find the EXACT problem, and not even a method to gain some download speed.

The only information I could find online are:

• Several users complain about the fact that Qualcomm Atheros AR8121/AR8113/AR8114 have a problem with SMB communication speed, in ONE direction, like me. I noticed that in all cases reported on internet, the Atheros harware was able to receive SMB full speed, but it was NOT able to send SMB at a decent speed.
• The problem seems connected to the Linux/Kernel driver atl1e for Qualcomm Atheros AR8121/AR8113/AR8114. If the same harware is used with Free BSD or with Windows, the problem does not happen. (I have not tried - not in my interest)
• Only the SMB protocol seems to be affected by this issue. In some cases, some users complains also low performance with NFS protocol, but this is not my case.

Some externale references, if someone in the future will find this thread:

• stackexchange - atl1e-interrupt-coalescing-ringbuffer-offloads-settings
  
• Developers try to find some bugs into linux driver ATL1E
  
• Another user with the same issue (for example only)
  

Conclusion:

if you have a Qualcomm Atheros AR8121/AR8113/AR8114 hardware and you find it very slow to send data, buy a new Network Card getting advice from experts.

Thank you all for your support and for your time.