SMB encryption with OMV using AES-NI?

  • Hi folks,


    Is there any way I can install Samba 4.7.0+ so as to get hardware offloas AES encryption of SMB? I'm using an Intel NUC with a Celeron 3050 for OpenMediaVault, and get 112MB/s without SMB encryption, and only 40MB/s with it enabled.

  • Out of curiosity:

    • With which SMB version is the client connecting? (might matter)
    • Is the bottleneck the server or maybe (also) the client?

    Can you provide testparm and smbstatus command output with the client connected?


    Have you checked with htop for example whether one CPU core on the server is maxing out?

    Clients are all Windows 10 and connecting with SMB3. I have a Samba server running on a VM which has four i7-6950X cores attached to it and it's maxing out the line (102MB/s).


    The bottleneck is not the client as it's getting the aforementioned 102MB/s from the VM server.


    Testparm output:


    smbstatus output (with encryption disabled/not enforced):




    HTOP with SMB encryption disabled:
    SMB3-cleartext.png



    HTOP with SMB encryption enabled (note, it does peak and hold at 100% for much of the time):
    SMB3-encrypted.png

  • The bottleneck is not the client as it's getting the aforementioned 102MB/s from the VM server

    Well, with encryption enabled it could've been the client too (since both ends of the connection need to use crypto functions) but your htop output points to the server's single-threaded CPU performance being the bottleneck (and no AES-NI in use of course).


    Since you're experienced it might be an idea to set up another VM with OMV5 (relying on Debian Buster and Samba 4.9) and give this a try. I wanted to test this already days ago but am running a bit out of time right now (and can't test directly since away from my OMV5 test VM and accessing through VPN):

  • Well, with encryption enabled it could've been the client too (since both ends of the connection need to use crypto functions) but your htop output points to the server's single-threaded CPU performance being the bottleneck (and no AES-NI in use of course).
    Since you're experienced it might be an idea to set up another VM with OMV5 (relying on Debian Buster and Samba 4.9) and give this a try.

    Better not be my client - it's an i7-8700k running at 5Ghz! :)


    Anyway, I'm happy to try OMV5 as this host isn't really in production use yet.

  • I'm happy to try OMV5 as this host isn't really in production use yet

    Please report back whether you can see transport encryption benefitting from AES-NI. Personally interested in this since in 2019 we have to switch with a lot of installations away from AFP/Netatalk/Helios to Samba (even dealing with Windows clients again after a decade).

  • Please report back whether you can see transport encryption benefitting from AES-NI. Personally interested in this since in 2019 we have to switch with a lot of installations away from AFP/Netatalk/Helios to Samba (even dealing with Windows clients again after a decade).

    Yeesh. OMV5 kernel panics on the NUC 3050 hardware when I boot the ISO. :-/

  • Just did a quick test with macOS 10.14.5 as SMB client (SMB3_02). AES-128-CCM first vs. no encryption:


    Code
    Service pid Machine Connected at Encryption Signing
    ---------------------------------------------------------------------------------------------
    ext4 11004 192.168.21.91 Wed May 22 11:55:55 2019 CEST AES-128-CCM AES-128-CMAC
    ext4 12864 192.168.21.91 Wed May 22 12:08:08 2019 CEST - -


    Encrypted:


    Bildschirmfoto 2019-05-22 um 12.26.42.png


    vs. unencrypted:


    Bildschirmfoto 2019-05-22 um 12.34.15.png


    Clearly a client side issue. I need to find a Windows box there... the server setup is not the bottleneck (up to 500 MB/s with 10GbE)


    Edit: Found the culprit: not encryption is the bottleneck but SMB packet signing. As test I created /etc/nsmb.conf on the Mac with enforcing signing by setting signing_required=yes and now get similar slow speeds as with encryption/signing:


    Code
    Service pid Machine Connected at Encryption Signing
    ---------------------------------------------------------------------------------------------
    ext4 14549 192.168.21.91 Wed May 22 12:23:20 2019 CEST - AES-128-CMAC

    Bildschirmfoto 2019-05-22 um 12.51.31.png

  • Now a test with OMV5 against a virtualized Win10 Pro system running on the same vSphere server:
    Bildschirmfoto 2019-05-22 um 16.48.25.png


    Both OMV5 and the Win10 VM have 2 vCPUs configured. Connection is established with signing and encryption enabled (SMB3_11 version):

    Code
    Service pid Machine Connected at Encryption Signing
    ---------------------------------------------------------------------------------------------
    ext4 6038 192.168.21.116 Wed May 22 16:07:38 2019 CEST AES-128-CCM AES-128-CMAC


    Samba --> Windows:
    Samba_to_Windows.png


    Windows --> Samba:
    Windows_to_Samba.png


    On the OMV server CPU utilization was identical both times, I use the values from an iostat 30 call:


    Code
    avg-cpu: %user %nice %system %iowait %steal %idle
    4.97 0.00 1.83 0.84 0.00 92.36
    45.67 0.00 12.70 6.49 0.00 35.15
    40.10 0.00 10.59 9.19 0.00 40.12
    42.78 0.00 11.76 4.99 0.00 40.47
    32.21 0.00 7.98 3.58 0.00 56.23
    1.15 0.00 1.02 0.18 0.00 97.64

    So we're talking about 60% CPU utilization with 2 CPU cores. To me this looks like at least on the Samba side encryption is making use of AES-NI while with that high CPU utilization on the Win10 Pro machine I'm not entirely sure but according to Coreinfo AES-NI support is there:


    Bildschirmfoto 2019-05-22 um 17.05.35.png

  • And another test again with Win10 but this time without encryption/signing:


    Samba --> Windows:
    SMB_Samba_to_Windows.png_unencrypted.png
    Still high CPU utilization in Windows but better performance. CPU utilization of the dual-core Samba server below:

    Code
    avg-cpu: %user %nice %system %iowait %steal %idle
    0.99 0.00 0.95 0.03 0.00 98.03
    1.39 0.00 6.72 3.10 0.00 88.79
    1.77 0.00 9.97 2.49 0.00 85.77
    2.01 0.00 22.18 8.74 0.00 67.08
    1.59 0.00 17.77 11.20 0.00 69.44
    1.43 0.00 12.78 8.66 0.00 77.12
    1.39 0.00 1.10 0.42 0.00 97.09


    Windows --> Samba:
    SMB_Windows_to_Samba_unencrypted.png


    And CPU utilization at the Samba server:


    Code
    avg-cpu: %user %nice %system %iowait %steal %idle
    0.54 0.00 0.54 0.20 0.00 98.72
    2.47 0.00 15.08 4.49 0.00 77.95
    3.78 0.00 36.35 11.35 0.00 48.52
    1.58 0.00 4.71 1.28 0.00 92.43
    1.19 0.00 0.94 0.00 0.00 97.87


    TL;DR: at least with OMV5 (Samba 4.9) SMB encryption runs AES-NI accelerated if the CPU supports it.


    But there is a price to pay: with encryption enabled the whole crypto stuff triggers constant switching between kernel and userspace, see the huge %user percentage with encryption vs. no encryption below where the whole network <--> storage thing runs inside the kernel.


    Code
    crypto: %user %nice %system %iowait %steal %idle
    45.67 0.00 12.70 6.49 0.00 35.15
    40.10 0.00 10.59 9.19 0.00 40.12
    42.78 0.00 11.76 4.99 0.00 40.47
    none: %user %nice %system %iowait %steal %idle
    1.77 0.00 9.97 2.49 0.00 85.77
    2.01 0.00 22.18 8.74 0.00 67.08
    1.59 0.00 17.77 11.20 0.00 69.44
  • Thanks so much for your testing effort here. I'll look at the Debian install with OMV installed on top, because whilst I won't be copying files using SMB every day, when I do need to it's generally large files and being an InfoSec guy I'm a bit of a stickler for transport encryption.

  • being an InfoSec guy I'm a bit of a stickler for transport encryption

    Well, 'best practices' in certain environments demand transport encryption. I'm about to migrate a few older AFP/SMB server installs to Samba this year and most probably we'll decide to differentiate between shares with sensitive data and others that don't need encryption. With Samba (and OMV) it's possible to define smb encrypt = auto in the global section and then smb encrypt = required per share so it's possible to adjust behavior on a per share and on a per client basis.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!