Another update with regard to the 'cheap and energy efficient single drive NAS' journey. Xalius, another dev, tested yesterday with ROCK64 and a rather performant SSD (Samsung 850 Pro with 512GB, this drive is bottlenecked by SATA 3.0 that means on a SATA port it will exceed 520 MB/s with sequential workloads even with rather small block sizes, also random IO performance is that high that numbers aren't different than sequential numbers). ROCK64's companion was a JMS578 disk enclosure, UAS capable of course.
We're talking about +380MB/s on average with UAS (full log here: http://pastebin.ubuntu.com/25064848/). This is close to the maximum what we can get with USB3 SuperSpeed anyway (400MB/s or to quote the specs: "At a 5 Gbps signaling rate with 8b/10b encoding, the raw throughput is 500 MBps. When link flow control, packet framing, and protocol overhead are considered, it is realistic for 400 MBps or more to be delivered to an application.")
Please remember: This is a $25 single board computer featuring an ARM SoC made for TV boxes (Rockchip RK3328). The CPU cores are somewhat underpowered compared to any recent x86 box but USB3 performance is simply impressive here and IMO we don't need any more storage tests wrt RK3328 USB3 host performance (well, most probably prefixing benchmark tests with 'ionice -c1 ' will further improve the 380 MB/s above but that's just tuning numbers).
A prerequisit for such impressive USB performance is 'USB Attached SCSI' (UAS) of course. It's way more efficient than the older USB Mass Storage protocol. And this does not only result in better performance but also less CPU utilization. To get an idea I simply tested with and without UAS (see [1] how to disable this on ROCK64). I used the usual iozone test but only with 2 blocksizes: 4KB and 16MB. Performance differs a lot (not that important if you want to use HDDs instead):
random random
write rewrite read reread read write
UAS 4K 18129 23421 27535 23642 15982 17115
Mass Storage 4K 13410 14787 16924 16926 11617 10639
UAS 16M 217132 219776 354657 354838 353728 183282
Mass Storage 16M 149317 163379 180475 180800 180795 116282
It's obvious that Mass Storage performance is much worse compared to UAS (especially with the huge blocksize -- since my test SSD is cheap crap normally the sequential write difference would also be much higher). But as already said: not that important if it's about using spinning rust (HDDs). So let's look at CPU utilization (I let an 'iostat 5' run in another terminal).
First CPU utilization for 4K block size:
4K UAS:
avg-cpu: %user %nice %system %iowait %steal %idle
write 0.09 0.00 17.85 4.25 0.00 77.82
rewrite 0.10 0.00 9.43 4.86 0.00 85.62
read 0.10 0.00 7.60 5.68 0.00 86.62
reread 0.67 0.00 14.42 10.03 0.00 74.88
rand read 0.10 0.00 1.05 14.97 0.00 83.87
rand write 0.00 0.00 1.11 10.09 0.00 88.80
4K Mass Storage:
avg-cpu: %user %nice %system %iowait %steal %idle
write 0.00 0.00 29.81 7.43 0.00 62.75
rewrite 0.00 0.00 21.16 9.95 0.00 68.89
read 0.29 0.00 22.65 10.84 0.00 66.22
reread 0.97 0.00 20.66 10.96 0.00 67.41
rand read 0.00 0.00 17.60 12.08 0.00 70.32
rand write 0.00 0.00 20.10 9.60 0.00 70.30
Alles anzeigen
I'm not entirely sure but AFAIK we should not rely on %iowait (and therefore %idle too) but can only have a look at %system. Then CPU utilization with Mass Storage is multiple times higher than with UAS on average and if we look at random IO then it's even 15 times higher). So if it's about random IO with small record/block sizes then UAS is way more efficient wrt CPU utilization (which is important since we should keep in mind that RK3328 is a quad core CPU and %25 utilization means one single CPU core is entirely busy and might bottleneck storage performance already).
With 16 MB block size it looks a lot better for Mass Storage:
16M UAS:
avg-cpu: %user %nice %system %iowait %steal %idle
write 0.00 0.00 5.97 20.00 0.00 74.03
rewrite 0.00 0.00 3.15 24.19 0.00 72.66
read 0.45 0.00 10.05 16.74 0.00 72.76
reread 0.63 0.00 9.95 16.65 0.00 72.76
rand read 0.53 0.00 8.28 17.99 0.00 73.20
rand write 0.00 0.00 3.60 22.09 0.00 74.32
16M Mass Storage:
write 0.00 0.00 11.20 23.73 0.00 65.07
rewrite 0.00 0.00 10.67 24.53 0.00 64.80
read 0.27 0.00 10.63 23.68 0.00 65.42
reread 0.27 0.00 9.76 13.38 0.00 76.58
rand read 0.36 0.00 9.85 13.37 0.00 76.42
rand write 0.00 0.00 5.08 20.41 0.00 74.50
Alles anzeigen
The larger the block size gets the less UAS has an advantage wrt lower CPU utilization. Which is understandable based on how UAS works (see link above).
[1] How to set USB quirks (eg. for external Seagtate disks) or disable UAS on ROCK64 OS images:
Let's take disabling UAS as a somewhat stupid example (you don't want to disable UAS since it's great!). It's necessary to create a file called eg. /etc/modprobe.d/disable-uas.conf (filename doesn't really matter) with the following contents:
(USB vendor and product ID have to match of course, check 'lsusb' output). And then you need to execute 'update-initramfs -u' since otherwise your settings won't be applied (initrd)