10GB Ethernet SMB Settings (extra options)

  • If you have a good working set of SMB extra options for 10GBe (10 Gigabit Ethernet) please post them.

    I would love to see what everyone is working with for these options because I cannot seem to extract the full 10GB out of my homelab.

    (More like 300MB/s - 400MB/s)

    I have 10GB NICs 2x 10GB Switches all Cat6-Cat8 Cables and patch cables.


    Anyway, please share your extra options if you are successfully running SMB at 10GB/s speeds.


    Thanks in advance.

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

  • Unless the design has changed and I am not aware of it, SAMBA is single threaded. As such your 300MB/s to 400MB/s is excellent and you will likely not be able to get much more out of it on a single file transfer.


    This does however mean that you could possibly get similar speeds from multiple parallel transfers, up to the limit of what the server can handle, assuming the clients are not throttling either.


    For example, we have SAN at the office that is 10Gbps iscsi. iscsi is faster than samba and not bound by the single threaded restriction, but as a rule, the speeds were restricted by the clients, ranging from around 600MBps to 1000MBps, OS dependant, and even with tuning (Windows on the lower end, Mac in the middle, and linux on the higher end),


    Even though a 10Gbps connection is theoretically able to do 10x the bandwidth of a 1 Gbps connection, the reality of the samba single threading, samba chattiness, and any other os related throttling issues, put your numbers right where I would expect to see them on a good to excellent setup.

  • Interesting, Ill have to look into this single thread thing because I dont fully understand it.lol

    I just figured running from my gaming machine with NVME to the NAS 10x SSD RAID6. It would do better.

    But again, Ill have to look into the single thread thing.


    Thanks for your input.


    Anyone else out there?

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

    • Offizieller Beitrag

    Anyone else out there?

    As I mentioned in your other thread, I am using the default options but I really don't use Windows and use nfs 99.9% of the time.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • This may improve performance a bit too.


    https://www.samba.org/samba/hi…20disabled%20by%20default.


    Basically it means putting this in the smb global extra options. ( I have added an extra 0 on the speed directive to make it 10Gb. You will have to adjust the IP address to your server)


    This is experimental, so use at your own risk.


    Code
    server multi channel support = yes
    interfaces = 192.168.1.42;speed=10000000000
  • @BernH Thank you for answering my actual request and giving me the opportunity to try new and exciting settings in SMB.

    Also, though

    @BernH I wonder why SMB is single threaded after all this time. Pretty much all modern computers have multiple cores and threads if I am not mistaken?? I mean, I am running machines that aren't even under manufacturer warranty at this point and even they have multi-thread multi-core. Makes me think there is a much better way. Like maybe the iscsi you mentioned OR NFS that @ryecoaaron runs with a different kernel.


    There has got to be a better way. Im just trying to get 10GB ethernet speeds here. How hard should this be if Linus Tech Tips is getting 100GB on his setups. Even before that he was getting 10GB speeds on their systems with very little effort it seemed. Maybe that hard stuff was on the B roll footage. And they werent using OMV either.. HMMM?? I digress.


    As I mentioned in your other thread, I am using the default options but I really don't use Windows and use nfs 99.9% of the time.

    @ryecoaaron I still value your input, I am also still reading up on the whole kernel situation and now NFS is on my to-read-about list.

    Care to give a 30 second tl;dr rundown on NFS?

    If not its all good.. you have probably posted it elsewhere anyway.haha


    Thanks for your input guys , cannot wait to try out your code @BernH.

    Is that your code in its entirety? Or just a snippet? Thanks.


    Hopefully there is more code to come from other users.

    I feel kinda nosey/jealous ;) of people getting 10GB speed transfers on OMV, since I believe it is possible and I want to believe it is possible on my machines too. 8o

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

  • @BernH your two lines of code seemed to help a good bit when run by themselves.

    I dont need any of the options I had before it seems. Those are below:

    Code
    protocol = SMB3
    min receivefile size = 65536
    write cache size = 524288
    use sendfile = true
    aio read size = 262144
    aio write size = 262144
    getwd cache = yes
    socket options = TCP_NODELAY IPTOS_LOWDELAY
    read raw = yes
    write raw = yes

    Using just the two lines you provided jumped me from 300MB/s-400MB/s closer to 375MB/s-415MB/s and even bouncing up to 603MB/s for a split second. So I am grateful to know there are more settings out there that can have a significant impovement on speeds. Yet, I am still just as confused as ever on what else to add to get me to a full 10GB transfer. But we are slowly getting there I think.



    Code
    server multi channel support = yes
    interfaces = 192.168.1.42;speed=10000000000           (adjust ip address to your server ip.)

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

    • Offizieller Beitrag

    I wonder why SMB is single threaded after all this time. Pretty much all modern computers have multiple cores and threads if I am not mistaken??

    SMB doesn't need multiple threads. Yes, modern computers all have multiple cores but I remember very working just fine on computers with a single core. A single core nowadays is fast enough to saturate 10GBe.


    And they werent using OMV either.. HMMM??

    You can blame OMV all you want but OMV is just creating a configuration file for the Debian samba package. You should try other samba servers like TrueNAS or whatever Linus Tech Tips was using and see if get faster speeds. I don't think you have ruled out your network and/or client(s). If you do get faster speeds, compare the smb.conf from the other solution with the one OMV generates. Any differences could easily be specified in extra options.

    Care to give a 30 second tl;dr rundown on NFS?

    Different file sharing protocol. Builtin to the linux kernel. Lighter weight than smb/cifs. Usually used with Linux or Mac clients. Much, much simpler config. More secure in my opinion.

    Hopefully there is more code to come from other users.

    You keep hoping to find something on this forum but between most OMV users only having gigabit and them being noobs, you are unlikely to get what you want. Since OMV is just creating a samba config file, you can look anywhere on the internet for smb speed tips and try them in the extra options box.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • @BernH Thank you for answering my actual request and giving me the opportunity to try new and exciting settings in SMB.

    Also, though

    @BernH I wonder why SMB is single threaded after all this time. Pretty much all modern computers have multiple cores and threads if I am not mistaken?? I mean, I am running machines that aren't even under manufacturer warranty at this point and even they have multi-thread multi-core. Makes me think there is a much better way. Like maybe the iscsi you mentioned OR NFS that @ryecoaaron runs with a different kernel.


    There has got to be a better way. Im just trying to get 10GB ethernet speeds here. How hard should this be if Linus Tech Tips is getting 100GB on his setups. Even before that he was getting 10GB speeds on their systems with very little effort it seemed. Maybe that hard stuff was on the B roll footage. And they werent using OMV either.. HMMM?? I digress.

    Samba is not ethernet, it is a rather old file sharing protocol that works with every OS. As such it is probably not as lean as it could be, like NFS for linux or even the old AFP for mac. It is also a lot "chattier" than many of the other protocols, but it is very tweakable. Like Aaron, for linux systems, I prefer to use NFS due to the performance and security. I never found NFS on a mac to be user friendly for the average user, so for them I would say AFP if you can and SAMBA if you can't.


    If you want to test your ethernet speeds, use iperf. That will tell you the maximum speeds your network can really do, which I suspect is what you are referring to with Linus. Hitting 100GBps for storage is almost impossible unless you are talking about using a massive array of SSD or nvme drives.

  • You can blame OMV all you want but

    Im not blaming OMV, just questioning it.



    between most OMV users only having gigabit and them being noobs, you are unlikely to get what you want.

    This is true, shouldn't the forum try to bridge the gap though. you have guys like you aaron and BernH that seem to be on a whole other level. and then you have the omv "noob" community. your word not mine. Anyway, if there were a way to get everyone on some middleground like not noobs but maybe not experts either... but just intermediate users. I think this could be possible with the proper tutorials and/ the like. IDK just thinking out loud here.

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

    • Offizieller Beitrag

    shouldn't the forum try to bridge the gap though

    How?


    Anyway, if there were a way to get everyone on some middleground like not noobs but maybe not experts either... but just intermediate users. I think this could be possible with the proper tutorials and/ the like. IDK just thinking out loud here.

    We have guides on just about everything. Have you looked at the guides forum or omv wiki or omv-extras wiki? Many noobs are noobs because they don't want to learn more and just want answers. Others are noobs because they aren't technical people. What you are asking for is impossible in my opinion especially when I am a horrible teacher. But the issue of getting enterprise performance on some networks is just really hard to help with when we don't have full access.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • What you are asking for is impossible in my opinion especially when I am a horrible teacher.

    aww... cmon man, you aren't that bad. lol .. nah really though you are pretty good at teaching or trying to teach those who don't want to be taught. Like you said:

    Many noobs are noobs because they don't want to learn more and just want answers.

    And that is the key right there.


    You try to guide people to the answers by letting them find the answers themselves. That is actually a great teacher methodology.

    IFF you had the correct pupils. IF only...


    I am willing to put in the time and learn because I enjoy it. Now do I like having an answer served up to me on a silver platter.. sure who doesn't. But at the same time I will take the notes you give me and run with them also.


    As in the case of SMB though. I feel as though I have hit a wall. Ive googled and forumed to death. This is one of those cases where I would just like to be spoon-fed an answer.haha I will be more than happy to tell you every single piece of hardware in my personal LAN OR any other info to help give you "full access" if that is even possible remotely. If it isn't possible, I will keep on keeping on just without my goals being met unfortunately.


    But the point I was trying to make is that you actually are a good teacher. And we are lucky that we have you. And we are lucky that you are so interactive/active in the forums.


    Lastly.. and oddly enough I may or may not have not looked at the "guides forum or omv wiki or omv-extras wiki".... Well I dont know if I have. I have looked at so many pages on SMB and OMV it has all kinda turned to mush between my ears at this point. I couldn't tell you what I have and have not looked at specifically. It would be a long list though. Maybe I should get back to basics and start over in my quest to conquer 10GBe SMB instead of going to the farthest of reaches or the most exotic answer selections. In my experience the answer is the most obvious one or it is staring you in the face.

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

    • Offizieller Beitrag

    I am willing to put in the time and learn because I enjoy it.

    This forum would be so much better if everyone was that way.

    I will be more than happy to tell you every single piece of hardware in my personal LAN

    Unfortunately, i can't look at a list of hardware and tell what settings need to be in place. I would have to try things through trial and error.

    help give you "full access" if that is even possible remotely.

    I wish I had more time to do stuff like that. I do stuff like that all day long at work and when I get home (especially when it is nice out), I just want to be outside. usually with a beer in my hand.

    I will keep on keeping on just without my goals being met unfortunately.

    There are other options. You could try different 10 GBe cards or cables or switch. I have mellanox connect x2 cards from ebay, a D-link DGS-1510-28X switch, and cheap DAC cables from amazon.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • when I get home (especially when it is nice out), I just want to be outside. usually with a beer in my hand.

    I totally get it!

    Chenbro NR12000 32GB Ram - Xeon E2-1230 V2 @ 3.3GHz - OMV6

    Supermicro X9DRI-LN4F+ 128GB Ram - 2x Xeon E5-2670 V2 @ 2.5GHz - OMV6

    2x Wiwynn Lyra SV315 10SFF 32GB Ram - 2x Xeon E5-2630 v2 @ 2.60GHz - OMV6

    2x Lenovo EMC PX12-400R 32GB Ram - Core i3-3220 @ 3.3GHz - OMV6

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!