RAID 5 slows down after a few weeks

  • I don't know how to troubleshoot this without buying expensive parts, like a RAID controller. Even if I did that, it might take so long (three weeks) for the problem to reoccur that I probably won't be able to use the store return windows. My transfer speeds are fine for weeks and then slow down to a fluctuating 20 to 30 mbps. Then I turn the NAS off for five seconds and turn it back on and I can transfer at a steady 113 mbps again. Is there anything in the diagnostics that should give me some idea of what the problem is?


    nas.jpg


    That gap on the right was where I turned off the power. It happened even before I added this SATA expansion card (https://www.newegg.com/p/1FR-00BS-000D3?Item=9SIAX3GETV7720) and expanded from a three drive array to four drives. Meaning it was the same using both the motherboard SATA ports and the card.


    Specs:


    Ryzen 3 1200

    GIGABYTE GA-A320M-S2H

    EVGA 400 N1, 400W

    Corsair Vengeance LPX 2x8GB DDR4 2400MHz

    Samsung SSD 840 EVO 120 GB (system drive)

    4x WD Red 12 TB (RAID 5)

    + that SATA expansion card linked above

  • Agricola

    Hat das Thema freigeschaltet.
  • Agricola

    Hat das Thema freigeschaltet.
  • My transfer speeds are fine for weeks and then slow down to a fluctuating 20 to 30 mbps.


    From reading "transfer speed" I assume network (what kind?) and a client are involved.

    what is the reasoning for assuming the root cause is on the server?

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • From reading "transfer speed" I assume network (what kind?) and a client are involved.

    what is the reasoning for assuming the root cause is on the server?


    I'm accessing the files through Windows 10 on my home network. The previous time, I reset the router, which didn't help. Another time, I was able to fix it by removing the network switch that goes to my PC, which solved it that time.


    NAS > router > network switch > PC


    I just kind of assumed it was the NAS this time because it was also happening on a computer in another room, which didn't have that problem the previous time, when I still used the switch in my room. I'll check the network when it happens again in probably another couple of weeks.


    Someone else told me it might be the power supply, but I feel like 400 Watts should be plenty for four hard drives, a CPU, SSD, three fans and two DDR4 sticks. Yesterday, I turned off Monitoring in the Power Management settings of OMV, believing that maybe the thing automatically selecting the "appropriate CPU levels" might be causing it. Is that definitely not related?

  • NAS > router > 2 PCs ?

    I assume you are using a wired LAN, no wireless networking involved at all?


    1.) I'm asking because what happened to me was that a laptop connected to LAN cable occasionally used slow WiFi due to a BIOS issue involving hibernation.
    2.) The second PC was switched off at the time the issue occurred? Otherwise the (usually) tiny CPU in router could be the root cause.
    Since I switched to a recent router with dual core CPU, my users get much better performance.

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • I assume you are using a wired LAN, no wireless networking involved at all?


    1.) I'm asking because what happened to me was that a laptop connected to LAN cable occasionally used slow WiFi due to a BIOS issue involving hibernation.
    2.) The second PC was switched off at the time the issue occurred? Otherwise the (usually) tiny CPU in router could be the root cause.
    Since I switched to a recent router with dual core CPU, my users get much better performance.


    Yeah, a wired LAN. Neither of the computers has WiFi.


    No, the other PC was being used by someone.


    Perhaps it is the router. Makes me regret restarting the NAS before testing/resetting the network. I just didn't want to bother the other person using his computer. Will just have to see again in a couple of weeks. Seems to happen sooner when I copy or remux many GB of data in a short period.

    • Offizieller Beitrag

    Perhaps it is the router

    Possibly, this is something I never do, I have a switch all my ethernet connections are connected to the switch with a single connection from the router to the switch to give internet access.


    Whilst most modern router ethernet connection should function as a switch my experience is that most of the time they appear to act as a hub, which means each ethernet connection gets 'polled' when data is transferred.

  • Possibly, this is something I never do, I have a switch all my ethernet connections are connected to the switch with a single connection from the router to the switch to give internet access.


    Whilst most modern router ethernet connection should function as a switch my experience is that most of the time they appear to act as a hub, which means each ethernet connection gets 'polled' when data is transferred.


    Interesting. Okay, then I'm putting my other switch to use.

    nas2.jpg


    I don't have Ethernet cables long enough to make it less complicated. Yes, I had the NAS plugged right into the router before. Need to just wait and see.

    • Offizieller Beitrag

    I don't have Ethernet cables long enough to make it less complicated

    This might be OK, I have a similar setup except my OMV box is in my office, but I run a cable from the office to a second switch in the loft space which in turn has ethernet connections to the downstairs. This is not ideal as one is putting in another pathway, but being on a small scale it generally works.


    A bad scenario of this was when I was asked to do a school survey, what someone had done was to 'daisy chain' 8 switches in what effectively was a loop just so they could all access the server and the internet. Needless to say the signal got worse the further away a computer was from the server due to the loop/hops the signal was trying to negotiate.

  • THis morning the NAS was completely inaccessible, and it's not the first time. Resetting the network didn't help; only turning off the NAS resolved it. Is it possible that this device my brother uses to charge all 120-something batteries on his bicycle sent bad magnetic signals through the case of the NAS? He keeps it right beside the NAS.


    DSC01670.jpg

  • I'd guess you are better positioned than us trying to verify this hypothesis

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • I don't think it's the network. I shut it down for twenty minutes and even tried plugging the NAS into the other network switch.


    router > network switch x NAS > my computer


    Instead of


    NAS > router > network switch > computer


    And still the only thing that fixed it was restarting the NAS. It's frustrating that there seems to be no way to tell in the OMV menu if a hard drive is bad. Before I restarted the NAS, it kept making a noise every fifteen to forty seconds that I haven't been able to find on YouTube so far. It's far less frequent after the restart, which makes it harder to troubleshoot. I thought of plugging only one hard drive in at a time to find out which of them is making that noise, but how can I do that if it takes sometimes days or weeks for the NAS to slow down? I wouldn't have used RAID 5 if I had known it was going to be this much trouble.


    So, what can I do? Just restart whenever it slows down and wait for the bad drive to die?

  • Actually, after the restart it IS still frequent enough that I can test it? Here's the sound:


    Externer Inhalt www.youtube.com
    Inhalte von externen Seiten werden ohne Ihre Zustimmung nicht automatisch geladen und angezeigt.
    Durch die Aktivierung der externen Inhalte erklären Sie sich damit einverstanden, dass personenbezogene Daten an Drittplattformen übermittelt werden. Mehr Informationen dazu haben wir in unserer Datenschutzerklärung zur Verfügung gestellt.


    So I can try plugging the drives in one at a time to see which of them is making that noise. But is there any way to confirm the slowdowns are a mechanical issue?

  • Took the NAS to my room where I could monitor it better. Whenever that noise came, the transfer speed slowed down to practically nothing. So I do have to RMA. Never had to do that with a hard drive in my life.

  • Figured the noise and the read speeds slowing to nothing temporarily was probably one of the hard drives rebooting due to insufficient power, so I plugged the NAS into my PC's PSU to test it my theory. The noise stopped. Then I went and bought another Corsair CX650M at Best Buy and replaced the 400 watt EVGA with it. Just watched a movie over the network for the first time in weeks, without any stutter and freezing. Hopefully this fixes the other issue that I've had far longer (a year) of the NAS severely slowing down after days or weeks too, but time will tell. My speeds since replacing the PSU have been a steady 112 to 113 mbps and sometimes fluctuating between 93 and 107.

  • Happening again. My transfer speeds fall from around 113 Mbps to 3 Mbps, then stay at about 20 briefly, then jump back up.


    To reiterate, I have four 12 TB hard drives in RAID 5, controlled by an SSD. I'm doing SMART tests on all of them, which will take 22 hours, but those tests haven't told me anything useful in the past. The SATA expansion card that they are all connected to is only PCIe 2, so 500 Mbps divided between the five storage devices. I wanna upgrade to a PCIe 3 SATA card. Unfortunately, the motherboard (GIGABYTE GA-A320M-S2H) only has one PCIe 3 slot, and because Ryzen CPUs have no integrated graphics I can't access the BIOS without a graphics card (PCIe 3). RAM is 2 x 8 GB DDR4.


    Started happening again probably not an hour after uploading 4 GB of videos to the internet. That took about two hours, so I'm wondering if those crappy upload speeds being extended so long put some stress on the array. Had no problems for a month, after replacing the power supply.


    What would you do, without spending hundreds of dollars? How would you troubleshoot it?

    Hoping it's just a bad SATA cable, but doubt it.

  • Someone told me to read my dmesg and /var/log/. And to "dd some big file to /dev/null," whatever that means, to see if it's the network or hard drive.


    I barely remember how we set this up. When I plug the monitor in, it asks for my username and password, but won't let me type in the password. Says, "To access the system visit the OMV web control panel," followed by six digits of numbers and letters, a colon and the IP address. Talking about this, I assume:

    omv.jpg

    I don't see anywhere to enter those Linux commands. I know he must have meant directly, meaning not over my browser, but I can't do anything directly because of the passwords not being entered.

    • Offizieller Beitrag

    How would you troubleshoot it?

    The only way you're going to narrow this down is by a process of elimination, which is painfully slow, tedious and frustrating, having replaced the power supply and got a temporary fix you're now into the realms of -> network, drives, sata cables, power cables to the drives, sata ports on m'board, m'board. The problems you're having could point to hardware degradation somewhere on the system and there will be no quick fix, unless someone comes up with a 'magic wand' command to point to the issue.


    As far as logs go, in the GUI under diagnostics there is system log that may be show something, dmesg could be of help, I would suggest doing a search for using dmesg to find errors it's what I would have to do.


    When I plug the monitor in, it asks for my username and password, but won't let me type in the password

    That's the console directly on OMV, you will see a login prompt, user name would normally be root -> hit enter, you're then asked for the root password this is the password that was created during the installation.

    ___________________________________________________________________________________________________


    This is going to be frustrating having gone through something similar quite some time ago, the system would 'throw' a hard drive, I tried everything I could think of, but it still did it, but I could remove the drive, erase it with dban put it back in (even on a different port) and it would work, three months later it would throw it again. Those same drives ran in my first OMV system without a problem for over 12 months until I upgraded them for larger drives.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!