RPi4 read/write performance for SMB shares

  • Last update: 17Feb2021

    To allow for a comparison of apples to apples and answer questions like "read or write rate is only #MB/sec on my hardware".


    Short summary:

    • A RPi4 will max out a 1Gbit/s wired network (common maximum speed in home and office networks)
    • performance bottleneck for clients accessing data on the NAS is the network hardware
      • best case for wireless network access via WiFi-4/801.11n is for Write ~ 10 MByte/s & Read ~ 13.5 MByte/s
      • best case for wireless network access via WiFi-5/801.11ac is Write ~ 18 MByte/s & Read ~ 20 MByte/s
      • random access of data is much slower between 1,3 MByte/s to 2 MByte/s for WiFi-4/801.11n & WiFi-5/801.11ac
    • investing in fast NAS hardware (i.e. SSD data disk, CPU, RAM) has low to no benefit to clients accessing the data, because network hardware is the real bottleneck
    • valid comparisons can only be performed by using the same tools. Details are documented below

    These are measured values from a:

    • server: RPi4 with 4GB RAM running
      • omv 5.5.19-1 (usul)
      • Debian 5.4.79
    • storage: 2x 6TB Toshiba HDWT360 HDD in Icy Box IB-RD3662-C31 /RAID1
      • connected via original USB3.1 cable of IB-RD3662-C31
      • formatted with ext4 file system
    • further configuration details at bottom

    Measurements:

    Performance of SMB share hosted on OMV RPi4 server connected via wired 1Gb Ethernet versus 2 WLAN routers using (802.11n) & (802.11.ac = WiFi5) protocol to a Windows 10 client.

    Measurement done with CrystalDiskMark 8 on a Windows 10 client.


    Many more benchmarks have been published. Worth mentioning are:

    - Jeff Geerling's blog titled "Building the fastest Raspberry Pi NAS, with SATA RAID"

    - optimum encrypted RAID5 settings for NAS with RaspberryPI 4 (here)


    1) Client and server connected via 1Gbit switch & wired Ethernet performance

    [Read Performance]

    Measurement MB/s IOPS
    SEQ 1MiB (Q= 8, T= 1) 93.0 88.7
    SEQ 1MiB (Q= 1, T= 1) 95.4 91.0
    RND 4KiB (Q= 32, T= 1) 43.2 10557.4
    RND 4KiB (Q= 1, T= 1) 8.4 2054.4


    [Write Performance]

    Measurement MB/s IOPS
    SEQ 1MiB (Q= 8, T= 1) 117.3 111.8
    SEQ 1MiB (Q= 1, T= 1) 111.4 106.2
    RND 4KiB (Q= 32, T= 1) 26.7 6518.8
    RND 4KiB (Q= 1, T= 1) 9.6 2337.6


    2) Connected via WiFi-4 /WLAN(802.11n) (WiFi router was old single core model)

    [Read]

    SEQ 1MiB (Q= 8, T= 1): 6.1 MB/s [ 5.8 IOPS]

    SEQ 1MiB (Q= 1, T= 1): 5.9 MB/s [ 5.6 IOPS]

    RND 4KiB (Q= 32, T= 1): 5.7 MB/s [ 1395.3 IOPS]

    RND 4KiB (Q= 1, T= 1): 1.3 MB/s [ 313.5 IOPS]


    [Write]

    SEQ 1MiB (Q= 8, T= 1): 11.7 MB/s [ 11.2 IOPS]

    SEQ 1MiB (Q= 1, T= 1): 10.1 MB/s [ 9.6 IOPS]

    RND 4KiB (Q= 32, T= 1): 9.7 MB/s [ 2356.0 IOPS]

    RND 4KiB (Q= 1, T= 1): 1.3 MB/s [ 317.6 IOPS]


    3) Connected via WiFi-4 /WLAN (802.11n) (WiFi router was new dual core model)

    [Read]

    SEQ 1MiB (Q= 8, T= 1): 14.2 MB/s [ 13.6 IOPS]

    SEQ 1MiB (Q= 1, T= 1): 13.6 MB/s [ 13.0 IOPS]

    RND 4KiB (Q= 32, T= 1): 11.3 MB/s [ 2765.9 IOPS]

    RND 4KiB (Q= 1, T= 1): 2.0 MB/s [ 485.6 IOPS]


    [Write]

    SEQ 1MiB (Q= 8, T= 1): 12.2 MB/s [ 11.6 IOPS]

    SEQ 1MiB (Q= 1, T= 1): 11.3 MB/s [ 10.8 IOPS]

    RND 4KiB (Q= 32, T= 1): 10.9 MB/s [ 2655.3 IOPS]

    RND 4KiB (Q= 1, T= 1): 1.9 MB/s [ 458.0 IOPS]


    3) Connected via WiFi-5 /WLAN (802.11ac) (WiFi router was new dual core model)

    [Read]

    SEQ 1MiB (Q= 8, T= 1): 38.4 MB/s [ 36.6 IOPS]

    SEQ 1MiB (Q= 1, T= 1): 19.9 MB/s [ 19.0 IOPS]

    RND 4KiB (Q= 32, T= 1): 21.1 MB/s [ 5140.6 IOPS]

    RND 4KiB (Q= 1, T= 1): 1.7 MB/s [ 426.0 IOPS]

    [Write]

    SEQ 1MiB (Q= 8, T= 1): 31.0 MB/s [ 29.6 IOPS]

    SEQ 1MiB (Q= 1, T= 1): 17.8 MB/s [ 17.0 IOPS]

    RND 4KiB (Q= 32, T= 1): 15.0 MB/s [ 3672.4 IOPS]

    RND 4KiB (Q= 1, T= 1): 1.7 MB/s [ 423.3 IOPS]



    ------------------------------------------------------------------------------



    Profile: Default

    Test: 64 MiB (x5) [Z: 22% (1237/5544GiB)]

    Mode: [Admin]

    Time: Measure 5 sec / Interval 5 sec

    Date: 2020/12/20 15:53:48

    Client OS: Windows 10 Professional [10.0 Build 19042] (x64)

    ------------------------------------------------------------------------------


    Server:

    root@nas:~# lsusb -t

    /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M

    |__ Port 2: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M

    /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M

    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M

    root@nas:~# uname -a

    Linux nas 5.4.79-v7l+ #1373 SMP Mon Nov 23 13:27:40 GMT 2020 armv7l GNU/Linux


    ------------------------------------------------------------------------------


    CrystalDiskMark 8.0.0 x64 (C) 2007-2020 hiyohiyo


    Crystal Dew World: https://crystalmark.info/

    ------------------------------------------------------------------------------

    * MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]

    * KB = 1000 bytes, KiB = 1024 bytes


    SMB Extra Options modified to:

    min receivefile size = 16384

    write cache size = 524288

    getwd cache = yes

    socket options = TCP_NODELAY IPTOS_LOWDELAY

    read raw = yes

    write raw = yes

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

    Edited 22 times, last by mi-hol: Short summary added ().

  • Good you asked, yes I did and updated first post

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • @metRo_ you may want to try an early version of a benchmark script to allow for comparison of results.

    https://github.com/mi-hol/DiskSpdAuto in case of issues please open a github issue there

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • Reaching "same speeds of Windows on osx" (MacOS) seems difficult according to some comments made on other sites

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • - optimum encrypted RAID5 settings for NAS with RaspberryPI 4 (here)

    I saw your cryptsetup benchmarks and noticed that I get almost 50% lower results on my Pi4. How can this be if no disks are even considered?

    Code
    cryptsetup benchmark @1,5 GHz
    # Tests are approximate using memory only (no storage IO).

    The second thing I noticed was that my Pi4 actually stays at 1.5GHz although I overclock in config.txt


    Code
    over_voltage=6
    arm_freq=2147
    gpu_freq=750

    OMV4 on ProLiant N54L + 5 x 3.5'' WD/Seagate HDs

    OMV5 on Raspberry Pi4

  • CrystalDiskMark 8.

    I've never done cryptsetup benchmarks , because only IO benchmarks are relevant for NAS

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • Kernel 5.10.11 has NOT introduced a SMB performance regression as was reported on RPi Forum & https://github.com/raspberrypi/linux/issues/4133 !


    My initial comment was a false conclusion due to Anti-Malware software that had changed between measurements on the Windows client!

    Repeated measurement on a Windows 10 Pro client.


    server: RPi4 with 4GB RAM running

    • omv 5.5.23-1 (usul)
    • Debian 5.10.11


    ------------------------------------------------------------------------------

    CrystalDiskMark 8.0.0 x64 (C) 2007-2020 hiyohiyo

    Crystal Dew World: https://crystalmark.info/

    ------------------------------------------------------------------------------

    * MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]

    * KB = 1000 bytes, KiB = 1024 bytes


    [Read]

    SEQ 1MiB (Q= 8, T= 1): 109.220 MB/s [ 104.2 IOPS] < 75982.32 us>

    SEQ 1MiB (Q= 1, T= 1): 103.986 MB/s [ 99.2 IOPS] < 10069.66 us>

    RND 4KiB (Q= 32, T= 1): 45.550 MB/s [ 11120.6 IOPS] < 2867.51 us>

    RND 4KiB (Q= 1, T= 1): 8.629 MB/s [ 2106.7 IOPS] < 472.22 us>


    [Write]

    SEQ 1MiB (Q= 8, T= 1): 117.207 MB/s [ 111.8 IOPS] < 70708.24 us>

    SEQ 1MiB (Q= 1, T= 1): 110.506 MB/s [ 105.4 IOPS] < 9466.85 us>

    RND 4KiB (Q= 32, T= 1): 13.929 MB/s [ 3400.6 IOPS] < 3330.61 us>

    RND 4KiB (Q= 1, T= 1): 6.246 MB/s [ 1524.9 IOPS] < 650.69 us>


    Profile: Default

    Test: 64 MiB (x5) [Z: 23% (1253/5544GiB)]

    Mode: [Admin]

    Time: Measure 5 sec / Interval 5 sec

    Date: 2021/02/10 17:40:56

    OS: Windows 10 Professional [10.0 Build 19042] (x64)




    Performance impact of installed Anti-Malware software (Tanium and SentinelOne) on Windows Enterprise client:

    ------------------------------------------------------------------------------

    CrystalDiskMark 8.0.0 x86 (C) 2007-2020 hiyohiyo

    Crystal Dew World: https://crystalmark.info/

    ------------------------------------------------------------------------------

    * MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]

    * KB = 1000 bytes, KiB = 1024 bytes


    [Read]

    SEQ 1MiB (Q= 8, T= 1): 28.937 MB/s [ 27.6 IOPS] <280809.89 us>

    SEQ 1MiB (Q= 1, T= 1): 28.101 MB/s [ 26.8 IOPS] < 37209.92 us>

    RND 4KiB (Q= 32, T= 1): 16.972 MB/s [ 4143.6 IOPS] < 7706.54 us>

    RND 4KiB (Q= 1, T= 1): 1.699 MB/s [ 414.8 IOPS] < 2403.04 us>


    [Write]

    SEQ 1MiB (Q= 8, T= 1): 43.206 MB/s [ 41.2 IOPS] <188604.44 us>

    SEQ 1MiB (Q= 1, T= 1): 35.021 MB/s [ 33.4 IOPS] < 29890.23 us>

    RND 4KiB (Q= 32, T= 1): 21.026 MB/s [ 5133.3 IOPS] < 6210.67 us>

    RND 4KiB (Q= 1, T= 1): 1.458 MB/s [ 356.0 IOPS] < 2254.01 us>


    Profile: Default

    Test: 64 MiB (x5) [Z: 23% (1253/5544GiB)]

    Mode:

    Time: Measure 5 sec / Interval 5 sec

    Date: 2021/02/10 16:28:37

    OS: Windows 10 Enterprise [10.0 Build 18363] (x64)

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

    Edited 4 times, last by mi-hol ().

  • Thanks a lot for the SMB Extra Options modified to:

    min receivefile size = 16384

    write cache size = 524288

    getwd cache = yes

    socket options = TCP_NODELAY IPTOS_LOWDELAY

    read raw = yes

    write raw = yes



    It did improve my system !:)

  • Didn’t improve anything on my system. I suspect that my bottleneck is the WIFI connection at home.

    What routers do people recommend for high speed SMB file access performance?

    OMV4 on ProLiant N54L + 5 x 3.5'' WD/Seagate HDs

    OMV5 on Raspberry Pi4

  • Any cheap WiFi6 router (prices in Germany start at 35Euro) will do the trick if your client accessing the data has a WiFi6 card too

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • Will this help any if HDD's are formatted NTFS? Or is there any tweak in your settings that will help the NTFS file system?


    On a side note, if I swapped out a Pi 4 2GB to a 4GB would it really make enough difference to do?

  • Valid questions but as I don't have the hardware I'm unable to answer.

    You are invited to provide benchmark results using your equipment.

    The setup I used is documented, hence it should be easy to reproduce.

    let me know if questions arise.

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • there are no WiFi6 USB cards available

    For what client computer are you looking for a WiFi6 NIC?

    I.e. laptops build since ~ 2006 have an internal mini PCIe slot for WiFi NIC and are easy to upgrade using for example a less than 20 Euro Intel AX200 card

    omv 5.6.5-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!