I'm having some throughput issues betwen OMV and a windows client when using mergerfs. Specifically, sustained reads are only 60-70MB/s
Here's the setup:
- OMV hosted on a VM connected to the internal 10Gb network on the hypervisor
- Windows client VM, also on the same hypervisor
Here are some things that I've tested to narrow down the issue specifically to mergerfs over SMB. All tests were performed with the VM read/write caches cleared:
- accessing files over SMB on one of the hard drives in the mergerfs volume yields a very stable read speed of 140 MB/s
- The raw read speed of the drive is 145 MB/s
- Direct read tests from the OMV server mergerfs volume yield read speeds of 108 MB/s
- These tests were done using dd if=big.file of=/dev/null bs=4k
- CPU usage jumps to 37% during the read
- Obviously, there's a decent performance hit here, but it's still enough to saturate 1Gb ethernet, which is how most of my other client machines connect
- Copying from the same mergerfs volume over SMB yields speeds that oscillate between 60-70MB/s
- CPU usage during these copies only have CPU usage at about 6%, which is pretty much the same as copying from the non-mergerfs share over SMB
What could it be specifically about mergerfs over SMB that causes such a slowdown?