SMB Speed starts out great but gets really slow (to around 20MB/s) a day after installation

Hi there!

So far I've been very happy with OMV and everything works as it should... well, almost.

When doing a fresh install, my SMB speeds are fantastic! Constantly maxing out between 90-110MB/s. Awesome! I could transfer 200GB of large files and ~100GB of many small files without a problem at all. 6 hours later, another ~500GB of files went through with a rock solid speed of 100MB/s as well.
But somehow, after a day or two after installing OMV, the SMB transfers are getting REALLY slow to about 5-20MB/s with even a few sudden drops to the KB/s range. Since I googled for around 5 hours and were getting nowhere, I just did a fresh install yesterday and everything was working fine again. Heavenly speeds. (I thought I must have somehow broken some kernel thing when I did an update previously).

But around 12h after the fresh install, the speeds are back to being horrendous.
I googled again for around 4 hours and I'm seriously getting frustrated so maybe someone can help me out here.

My configuration is:
Raspberry Pi 4B,
Root fs on SD card (64GB Sandisk Extreme UHS-I Class 10)
Raspian Buster Lite -> Install Script -> OMV
Data drives are 2x 4TB HDD's, connected via USB3 (external power) in a 2-bay enclosure
Gigabit LAN (working and recognized as such)

Since the configuration is working perfectly after fresh install, I can rule out a hardware issue, so it must be some configuration. What I did so far trying to resolve the issue:
- Rebooting OMV and PC (of course, lol)
- Checking Energy Management, tried different settings
- Tried every possible setting in smb.conf
- Disabled SMB 1.0 in Windows
- Tried different SMB versions
- Checking System Time synchronization of PC and OMV
- Switched from DHCP to static (although my router is assigning a static IP address anyway)
- Checking "top" -> CPU is almost at idle all the time, around 0.3%
- Checking Ram, using 0.1% of total RAM
- Checking "iotop" -> Nothing special
- (i'm sure I did some other things as well but these are the ones I remember at least)

In my mind, there can also be another two possible issues that I didn't check:
- Wrong SMB config (since I did shutdown and start my PC in between the "working good" and "working bad" cases)
- The Filetransfer is somehow getting cached on the SD card and getting transferred to the HDD (since the ~20MB/s is the write speed I get when writing directly to SD card via SMB)
(these are just "meh, could theoretically be possible" scenarios)

So... does anyone have a clue about what might be happening here? I'm getting crazy!

Thanks

Zitat von Adoby

I have no problems saturating GbE when transferring large files to/from my RPi4. However, I use NFS.

But if I add extra protocols like SSH or rsync, the speed goes down. Also if it is many small files rather than few large. Also if it is several parallel transfers.

if the SD card is involved in caching any of the file transfers to from your RPi4, then your OMV install is massively borked. So I doubt that is it.

Temperature problems?
Power problems?
The windows client in the other end is slow?

Alles anzeigen

Ah yeah, I should've mentioned: using ext4.

Funnily enough, temperature is the only thing I didn't check, but I'm guessing this won't be it since I've installed heatsinks and a fan on the RPi.
Power can't be the issue because it's a proper 3A PS.
Windows client is an i7 @4.4Ghz with 32 Gigs of ram, checked transfer from NVME SSD as well as HDD - makes no difference.

I'm completely clueless...

Still, will check temperature and try to disable ssh and see how it goes.

EDIT: Just checked and idle CPU is around 40°C. While transferring it stays rock solid @42°C, occasionally spikes up to 43°C but that's it.

Zitat von Adoby

If you reinstall, make sure to clone the SD card at once. And see if things get fast again after restoring the card.

Is there something else installed, on top of OMV? Did you change ANY settings at all before performance was reduced?

Try iperf to check bandwidth.

Try to boot the client from a live Linux distro and test Linux to Linux.

Also the SATA / USB interface in the enclosure might be incompatible with Linux.

No need to disable SSH, if you don't use it when copying files.

Alles anzeigen

Nope, nothing there. No Docker, no Portainer, no other software. Only OMV.
Sure, I did the usual stuff like adding users, adding shares and stuff but nothing fancy. Not even an update.

iperf test looked just like I suspected:

The JBOD enclosure might not be ideal for Linux but that still doesnt explain why I get full ~100MB/s for HOURS on end until this error occurs

EDIT: Will test on a Linux Live version now to see if smb is the problem. Will report back asap.

EDIT 2: Just booted from Linux Live (Ubuntu) and the problem was still there, although a bit better: 30MB/s write. Couldnt use NFS though and had to use SMB/CIFS. For some reason Ubuntu didn't like NFS

I think I might look into the USB Blacklisting thing and if that doesn't work I'm moving on.

My journey has been going on for almost a week now and I'm getting tired of it.
First, nextcloud: Good GUI and compatibility, excellent read/write speeds but the system (e.g. the whole data structure) is too fragile imho and somehow the CPU started going to 100% all the time because of some infinite loop in a php file and there's no patch besides manually editing the code. I could do that but I don't wanna trust the safety of my data with this kind of system.
Now OMV, which seems rock solid, with I/O from 1999 (at least in my case, lol).
I know a DIY NAS comes with a lot of work but this constant searching for the needle in a haystack is making me crazy.
Will try FreeNAS next.

Anyhow, I really really appreciate your help! Thanks so much for your answers and taking the time to help me out

Okay I just checked the UAS thing and it's not that either.

Driver for the enclosure and the HDD is set to
xhci_hcd/4p, 5000M and
usb-storage, 5000M

I'm running out of ideas now.

Zitat von thedarkness

I don't know if you replied with a joke, but I was serious. Maybe the SD was overheating - is the Flash Memory plugin installed?
@Kaliber45 you could try installing OMV on another media, like a USB stick, and see if it makes a difference.

Tested the "cold" pi (12+ hours off) and the result was the same.
Just did ANOTHER fresh install of OMV and I get constant 110MB/s write speed (read speed was always good) again. So it can't be the SD Card because it's usually very hot after formatting, writing, installing Armbian Buster etc.

So the only thing that could possibly have caused this are two settings regarding the HDD (which I didn't make now):
- Advanced Power Management (although setting it back after the bug occured didnt bring any difference previously)
- Spindown Time

After another 12 hours of searching, I think I'll officially give up. Still the same issue: Fresh install (just added a user to connect, NOTHING else) -> 110MB/s constant for 24h. But when I reboot the PI -> back to 20MB/s.