Hi folks,
Is there any way I can install Samba 4.7.0+ so as to get hardware offloas AES encryption of SMB? I'm using an Intel NUC with a Celeron 3050 for OpenMediaVault, and get 112MB/s without SMB encryption, and only 40MB/s with it enabled.
Hi folks,
Is there any way I can install Samba 4.7.0+ so as to get hardware offloas AES encryption of SMB? I'm using an Intel NUC with a Celeron 3050 for OpenMediaVault, and get 112MB/s without SMB encryption, and only 40MB/s with it enabled.
Out of curiosity:
Can you provide testparm and smbstatus command output with the client connected?
Have you checked with htop for example whether one CPU core on the server is maxing out?
Out of curiosity:
- With which SMB version is the client connecting? (might matter)
- Is the bottleneck the server or maybe (also) the client?
Can you provide testparm and smbstatus command output with the client connected?
Have you checked with htop for example whether one CPU core on the server is maxing out?
Clients are all Windows 10 and connecting with SMB3. I have a Samba server running on a VM which has four i7-6950X cores attached to it and it's maxing out the line (102MB/s).
The bottleneck is not the client as it's getting the aforementioned 102MB/s from the VM server.
Testparm output:
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Processing section "[backups]"
Loaded services file OK.
Server role: ROLE_STANDALONE
Press enter to see a dump of your service definitions
# Global parameters
[global]
server string = %h server
workgroup = MELNED
local master = No
log file = /var/log/samba/log.%m
logging = syslog
max log size = 1000
panic action = /usr/share/samba/panic-action %d
disable spoolss = Yes
load printers = No
printcap name = /dev/null
disable netbios = Yes
server min protocol = SMB3
smb ports = 445
pam password change = Yes
passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
passwd program = /usr/bin/passwd %u
restrict anonymous = 2
socket options = TCP_NODELAY IPTOS_LOWDELAY
dns proxy = No
idmap config * : backend = tdb
printing = bsd
create mask = 0777
directory mask = 0777
aio read size = 16384
aio write size = 16384
use sendfile = Yes
[backups]
path = /srv/dev-disk-by-label-backups/backups
hide special files = Yes
create mask = 0664
directory mask = 0775
force create mode = 0664
force directory mode = 0775
inherit acls = Yes
read only = No
Alles anzeigen
smbstatus output (with encryption disabled/not enforced):
Samba version 4.5.16-Debian
PID Username Group Machine Protocol Version Encryption Signing
----------------------------------------------------------------------------------------------------------------------------------------
16952 eluck users 192.168.0.100 (ipv4:192.168.0.100:59493) SMB3_11 - partial(AES-128-CMAC)
Service pid Machine Connected at Encryption Signing
---------------------------------------------------------------------------------------------
IPC$ 16952 192.168.0.100 Tue May 21 16:55:27 2019 AEST - -
backups 16952 192.168.0.100 Tue May 21 16:55:26 2019 AEST - -
Locked files:
Pid Uid DenyMode Access R/W Oplock SharePath Name Time
--------------------------------------------------------------------------------------------------
16952 1000 DENY_NONE 0x100081 RDONLY NONE /srv/dev-disk-by-label-backups/backups . Tue May 21 16:55:28 2019
16952 1000 DENY_NONE 0x100081 RDONLY NONE /srv/dev-disk-by-label-backups/backups . Tue May 21 16:55:28 2019
Alles anzeigen
HTOP with SMB encryption disabled:
HTOP with SMB encryption enabled (note, it does peak and hold at 100% for much of the time):
The bottleneck is not the client as it's getting the aforementioned 102MB/s from the VM server
Well, with encryption enabled it could've been the client too (since both ends of the connection need to use crypto functions) but your htop output points to the server's single-threaded CPU performance being the bottleneck (and no AES-NI in use of course).
Since you're experienced it might be an idea to set up another VM with OMV5 (relying on Debian Buster and Samba 4.9) and give this a try. I wanted to test this already days ago but am running a bit out of time right now (and can't test directly since away from my OMV5 test VM and accessing through VPN):
root@buster:/home/tk# smbstatus
Samba version 4.9.5-Debian
PID Username Group Machine Protocol Version Encryption Signing
----------------------------------------------------------------------------------------------------------------------------------------
6089 tk users 10.0.64.4 (ipv4:10.0.64.4:62921) SMB3_02 AES-128-CCM partial(AES-128-CMAC)
Service pid Machine Connected at Encryption Signing
---------------------------------------------------------------------------------------------
btrfs 6089 10.0.64.4 Tue May 21 09:59:36 2019 CEST AES-128-CCM AES-128-CMAC
Locked files:
Pid Uid DenyMode Access R/W Oplock SharePath Name Time
--------------------------------------------------------------------------------------------------
6089 1000 DENY_NONE 0x100081 RDONLY NONE /srv/dev-disk-by-path-pci-0000-03-00.0-scsi-0-0-1-0-part2/btrfs . Tue May 21 09:59:38 2019
root@buster:/home/tk# testparm
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Registered MSG_REQ_POOL_USAGE
Registered MSG_REQ_DMALLOC_MARK and LOG_CHANGED
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Processing section "[ext4]"
Processing section "[btrfs]"
Loaded services file OK.
Server role: ROLE_STANDALONE
Press enter to see a dump of your service definitions
# Global parameters
[global]
disable spoolss = Yes
dns proxy = No
load printers = No
log file = /var/log/samba/log.%m
logging = syslog
map to guest = Bad User
max log size = 1000
pam password change = Yes
panic action = /usr/share/samba/panic-action %d
passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
passwd program = /usr/bin/passwd %u
printcap name = /dev/null
server min protocol = SMB2
server string = %h server
socket options = TCP_NODELAY IPTOS_LOWDELAY
fruit:model = MacPro
fruit:aapl = yes
idmap config * : backend = tdb
aio read size = 16384
aio write size = 16384
create mask = 0777
directory mask = 0777
printing = bsd
smb encrypt = required
use sendfile = Yes
[ext4]
comment = ext4 partition on omv5 dataset
create mask = 0664
directory mask = 0775
ea support = No
force create mode = 0664
force directory mode = 0775
guest ok = Yes
hide special files = Yes
inherit acls = Yes
path = /srv/dev-disk-by-path-pci-0000-03-00.0-scsi-0-0-1-0-part1/ext4/
read only = No
store dos attributes = No
vfs objects = full_audit catia fruit streams_xattr
fruit:time machine = yes
fruit:resource = file
fruit:metadata = netatalk
fruit:locking = none
fruit:encoding = private
full_audit:priority = NOTICE
full_audit:facility = local7
full_audit:failure = none
full_audit:success = mkdir rename unlink rmdir pwrite
full_audit:prefix = %u|%I|%m|%P|%S
[btrfs]
comment = btrfs partition on omv5 dataset
create mask = 0664
directory mask = 0775
ea support = No
force create mode = 0664
force directory mode = 0775
guest ok = Yes
hide special files = Yes
inherit acls = Yes
path = /srv/dev-disk-by-path-pci-0000-03-00.0-scsi-0-0-1-0-part2/btrfs/
read only = No
store dos attributes = No
vfs objects = catia fruit streams_xattr
fruit:time machine = yes
fruit:resource = file
fruit:metadata = netatalk
fruit:locking = none
fruit:encoding = private
Alles anzeigen
Well, with encryption enabled it could've been the client too (since both ends of the connection need to use crypto functions) but your htop output points to the server's single-threaded CPU performance being the bottleneck (and no AES-NI in use of course).
Since you're experienced it might be an idea to set up another VM with OMV5 (relying on Debian Buster and Samba 4.9) and give this a try.
Better not be my client - it's an i7-8700k running at 5Ghz!
Anyway, I'm happy to try OMV5 as this host isn't really in production use yet.
I'm happy to try OMV5 as this host isn't really in production use yet
Please report back whether you can see transport encryption benefitting from AES-NI. Personally interested in this since in 2019 we have to switch with a lot of installations away from AFP/Netatalk/Helios to Samba (even dealing with Windows clients again after a decade).
Please report back whether you can see transport encryption benefitting from AES-NI. Personally interested in this since in 2019 we have to switch with a lot of installations away from AFP/Netatalk/Helios to Samba (even dealing with Windows clients again after a decade).
Yeesh. OMV5 kernel panics on the NUC 3050 hardware when I boot the ISO. :-/
OMV5 kernel panics on the NUC 3050 hardware when I boot the ISO
In the past the OMV ISOs couldn't cope with UEFI boot. Maybe this is still is an issue? Anyway: I succeeded installing Debian 10 RC1 and then Install OMV5 on Debian 10 (Buster)
Just did a quick test with macOS 10.14.5 as SMB client (SMB3_02). AES-128-CCM first vs. no encryption:
Service pid Machine Connected at Encryption Signing
---------------------------------------------------------------------------------------------
ext4 11004 192.168.21.91 Wed May 22 11:55:55 2019 CEST AES-128-CCM AES-128-CMAC
ext4 12864 192.168.21.91 Wed May 22 12:08:08 2019 CEST - -
Encrypted:
vs. unencrypted:
Clearly a client side issue. I need to find a Windows box there... the server setup is not the bottleneck (up to 500 MB/s with 10GbE)
Edit: Found the culprit: not encryption is the bottleneck but SMB packet signing. As test I created /etc/nsmb.conf on the Mac with enforcing signing by setting signing_required=yes and now get similar slow speeds as with encryption/signing:
Now a test with OMV5 against a virtualized Win10 Pro system running on the same vSphere server:
Both OMV5 and the Win10 VM have 2 vCPUs configured. Connection is established with signing and encryption enabled (SMB3_11 version):
Service pid Machine Connected at Encryption Signing
---------------------------------------------------------------------------------------------
ext4 6038 192.168.21.116 Wed May 22 16:07:38 2019 CEST AES-128-CCM AES-128-CMAC
On the OMV server CPU utilization was identical both times, I use the values from an iostat 30 call:
avg-cpu: %user %nice %system %iowait %steal %idle
4.97 0.00 1.83 0.84 0.00 92.36
45.67 0.00 12.70 6.49 0.00 35.15
40.10 0.00 10.59 9.19 0.00 40.12
42.78 0.00 11.76 4.99 0.00 40.47
32.21 0.00 7.98 3.58 0.00 56.23
1.15 0.00 1.02 0.18 0.00 97.64
So we're talking about 60% CPU utilization with 2 CPU cores. To me this looks like at least on the Samba side encryption is making use of AES-NI while with that high CPU utilization on the Win10 Pro machine I'm not entirely sure but according to Coreinfo AES-NI support is there:
And another test again with Win10 but this time without encryption/signing:
Samba --> Windows:
Still high CPU utilization in Windows but better performance. CPU utilization of the dual-core Samba server below:
avg-cpu: %user %nice %system %iowait %steal %idle
0.99 0.00 0.95 0.03 0.00 98.03
1.39 0.00 6.72 3.10 0.00 88.79
1.77 0.00 9.97 2.49 0.00 85.77
2.01 0.00 22.18 8.74 0.00 67.08
1.59 0.00 17.77 11.20 0.00 69.44
1.43 0.00 12.78 8.66 0.00 77.12
1.39 0.00 1.10 0.42 0.00 97.09
And CPU utilization at the Samba server:
avg-cpu: %user %nice %system %iowait %steal %idle
0.54 0.00 0.54 0.20 0.00 98.72
2.47 0.00 15.08 4.49 0.00 77.95
3.78 0.00 36.35 11.35 0.00 48.52
1.58 0.00 4.71 1.28 0.00 92.43
1.19 0.00 0.94 0.00 0.00 97.87
TL;DR: at least with OMV5 (Samba 4.9) SMB encryption runs AES-NI accelerated if the CPU supports it.
But there is a price to pay: with encryption enabled the whole crypto stuff triggers constant switching between kernel and userspace, see the huge %user percentage with encryption vs. no encryption below where the whole network <--> storage thing runs inside the kernel.
Thanks so much for your testing effort here. I'll look at the Debian install with OMV installed on top, because whilst I won't be copying files using SMB every day, when I do need to it's generally large files and being an InfoSec guy I'm a bit of a stickler for transport encryption.
being an InfoSec guy I'm a bit of a stickler for transport encryption
Well, 'best practices' in certain environments demand transport encryption. I'm about to migrate a few older AFP/SMB server installs to Samba this year and most probably we'll decide to differentiate between shares with sensitive data and others that don't need encryption. With Samba (and OMV) it's possible to define smb encrypt = auto in the global section and then smb encrypt = required per share so it's possible to adjust behavior on a per share and on a per client basis.
Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!