Raspberry Pi 4 and very slow file transfer speeds.

Quote from hrseldon

I'm running OMV on a Raspberry Pi 4 and I have two usb hard disk drives attached to it.

I'm sharing the files via Samba with a Windows 11 machine. One hard drive (a Seagate 6gb with its own power and USB hub) has transfer speed around 40mb/sec.

The problem is the 2tb WD drive. It can hardly reach 10mb/sec while writing. I connect it through the USB hub on the other drives. I changed it to a USB3 port on the RP4 itself and the speed didn't improve at all.

If I connect those drives to a Window 11 machine both can easily reach 110mb/sec and above.

So drives or their cables aren't the problem.

I converted the slow (WD 2tb) to EXT4 file format, but again speeds didn't improve at all.

I'm not sure where I should be looking. What may be the problem?

Display More

40mb is horrible, but 40MB, while not ideal, is not nearly as bad, particularly if wifi is in use. Note the b vs B. b is bits, B is bytes. there are 8 bits in 1 Byte, so so 40mb is only 5MB, but 40MB is 320mb. Please confirm bits or bytes, as it is a major distinction.

Also, for clarification, if you using wifi to connect, it will be significantly slower that a cable connection (usually in 1/10 to 1/20 kind of speed assuming gigabit (1000mb/s or 125MB/s) cable ethernet as the reference point as without any kind of wifi draft acceleration, you can only expect a maximim of 56mb/s from wifi, which is just above that 1/20 speed (1000/20=50). With draft acceleration, you might get around 320mb/s, which is that 40 MB/s

Someone that uses an RPi may be able to confirm if this is about normal, and if there are RPi related things that can be adjusted, but knowing the b vs B reference is crucial.

Quote from hrseldon

It is 10 MB/s, from the Windows 11 file transfer window.

The other harddisk on the same RP and same USB 3 port can transfer about 40MB/s, which is far from the top speed of the drive but at least acceptable.

You didn't answer about the wifi.

Additionally, if it is a lot of small files being copied, it will be a lot slower than one large file.

I don't use an RPi, and am full 1Gb/s cable connection. I can reach around 115MB/s to 120MB/s from my Win11 system, and may be able to do better if I wasn't using realtek NIC's, which is why I said someone that uses an RPi can comment on normal speeds.

As such I can't help much with your hardware config, but presenting the numbers and hardware usage correctly can make a big difference to the right RPi user to help.

If that is a single 1GB file and not a 1gb file (big B vs little b is still a major difference), you should have gotten much better than 10MB/s I would think.

However, since I am not a Pi user and don't know the normal speeds, I would suggest you do so dd testing on the drives and some iperf testing between the systems to see if there is a problem with the drive or the network connection. When you run these tests, it will show you the maximum read/write of the drive in question, and the maximum network performance between the systems. Depending on the traffic there would be some overhead subtracted from the maximum numbers, but they should give you a good reference point.

dd is in the debian linux install that omv runs on (need to use the cli to use it for this purpose), and iperf3 can easily be installed via the omv plugins or at the cli. There is a downloadable windows version of ipef3 as well as documentation on the usages of dd and iperf below.

https://www.cyberciti.biz/faq/howto-linux-unix-test-disk-performance-with-dd-command/

iPerf - iPerf3 and iPerf2 user documentation

GitHub - ar51an/iperf3-win-builds: iperf3 binaries for Windows. Benchmark your network limits.

iperf3 binaries for Windows. Benchmark your network limits. - ar51an/iperf3-win-builds

github.com

Quote from hrseldon

I did a few experiments. disconnected the drives and boot up that way, then connect them again and boot them again. etc.

I dont know what this changed my it started to transfer insanely fast. 120 MB/s from the computer and 170 MB/s between the hard disk on the RP4.

I'm not sure what happened but somehow, disconnecting all of them and connecting again solved the problem.

Starting Jellyfin, slows the drives again. Wonder what that got to do with anything. It accesses the NTFS drive. Maybe there is somekind of a conflict with the NTFS service. When transfering from the computer to that particular drive, it can be around 120MB/s, which is the max I get with it. But whenever a NTFS drive connected to RP involved, the transfers drop to 5-10MB/s. Highly likely NTFS service of Linux is involved.

If the file is like 200MB (byte) then transfer goes at the max speed of the drive 120MB/s, but if it is bigger than that like 700MB, then at the middle of the transfer speed starts to drop to 10MB/s. Something bogging the system, I checked the CPU usage, it seems ok.

Display More

NTFS is not a linux filesystem It will only cause you problems if you are using it for anything other that a way to transfer diles from the linux system to a windows system. It should never be used as permanent storage with linux. The reason for this is because it does not use linux ownership and permissions info, so only the linux root user is able to access them correctly. As such services like docker and samba that rely in the permissions to function correctly will have problems.

All drives used as permanent storage with linux should be in one of the formats that omv lets you format a drive as (ext4, xfs, btrfs, jfs, and f2fs - the late three I have never used, so I have no comment on them)