Reduce md array check

  • I have an OMV5 server with 2 x 10Tb disks in Raid 1 mode. Every first Sunday of the current month, the system launches a "check md array". (/etc/cron.d/mdadm). The check takes more than 24 hours and my server becomes very slow (average load: 1.67, max 2.43). A file transfer via ftp is 10 times slower during the check. I would like to reduce this task to only 1 check every 3 months and reduce the load with "nice". Can I suspend this task during a file transfer ?

    - NAS 1 : Debian 11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Pentium G3460 @ 3.50GHz

    - NAS 2 : Debian11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Core 2 Duo E8400

  • Edit : 72 h after the start of checking task, still checking... So i found this page https://community.ovh.com/t/ra…aille-des-disques/19179/8 (sorry, in french) were i can reduce to only 1 check every 3 month by modifiing /etc/cron.d/mdadm.

    - NAS 1 : Debian 11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Pentium G3460 @ 3.50GHz

    - NAS 2 : Debian11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Core 2 Duo E8400

    • Offizieller Beitrag

    i can reduce to only 1 check every 3 month by modifiing /etc/cron.d/mdadm.

    And OMV will change your edit. I recommend using rsync instead of raid 1. Raid 1 isn't back and most people don't need realtime sync.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I don't think you understand my problem. I don't need the mdadm task to check my raid every month. This task takes far too long and it prevents me from using my server normally.

    - NAS 1 : Debian 11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Pentium G3460 @ 3.50GHz

    - NAS 2 : Debian11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Core 2 Duo E8400

    • Offizieller Beitrag

    I don't think you understand my problem. I don't need the mdadm task to check my raid every month. This task takes far too long and it prevents me from using my server normally.

    Of course, I understand your problem. I'm just telling you that you won't be able to change it. OMV owns the configuration. So, even if you change it on the command line, OMV will change it back. So, since you can't change, I was recommending not using raid and reiterating that raid isn't back (not all users realize that).

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • So, if I understand you correctly, you don't recommend using disks in raid mode to store files in a nas ! It's really very strange, because many users use disks in raid mode !

    - NAS 1 : Debian 11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Pentium G3460 @ 3.50GHz

    - NAS 2 : Debian11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Core 2 Duo E8400

    • Offizieller Beitrag

    So, if I understand you correctly, you don't recommend using disks in raid mode to store files in a nas ! It's really very strange, because many users use disks in raid mode !

    I don't think that is what he is saying,.. but when it comes to raid 1, I will. I think raid 1 is more or less useless UNLESS you need 100% real time mirroring, which a vast majority of home users do not.


    Keep the drives independent, set up a simple rsync job, schedule it to run, 1, 2, 5 (or whatever) times a day.. and be done with it. I switched to using rsync vs raid 1 years ago, and wouldn't go back to raid 1.

  • Ok, I get it. The question is how to remove the raid 1 mode from the administration interface without losing my data. Aren't there metadatas on the disks to check that the data is identical ?

    - NAS 1 : Debian 11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Pentium G3460 @ 3.50GHz

    - NAS 2 : Debian11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Core 2 Duo E8400

    • Offizieller Beitrag

    you don't recommend using disks in raid mode to store files in a nas ! It's really very strange, because many users use disks in raid mode !

    And most users don't understand why they are using raid.... I don't use raid on my personal NAS at home but I do maintain hundreds of systems using raid at work. What does that tell you?


    The question is how to remove the raid 1 mode from the administration interface without losing my data. Aren't there metadatas on the disks to check that the data is identical ?

    You can't do it from the web interface. If you don't have another drive, you will have to create a degraded array by failing one drive, formatting it with ext4 (or another Linux filesystem), mount it, and rsync the degraded array to the new filesystem. once that is done, you wipe the array, format the other drive, and sync the two with rsync.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Your solution doesn't seem clear to me, I don't want to lose my data. I would rather try a solution like UnRaid.

    - NAS 1 : Debian 11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Pentium G3460 @ 3.50GHz

    - NAS 2 : Debian11 (bullseye) - OpenMediaVault 6.9.11-3 Shaitan - Kernel 6.1.0-0.deb11.13-amd64 - CPU Intel Core 2 Duo E8400

    • Offizieller Beitrag

    Your solution doesn't seem clear to me, I don't want to lose my data. I would rather try a solution like UnRaid.

    If you don't have a backup, no raid is a good solution. Not sure how to clear up my suggestion. Unraid is proprietary and not free or available on anything but Unraid. But even with unraid, you would have to re-format your drives losing your data.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I just got caught by the same RAID resync today (first Sunday of the month). I too would like to reduce the frequency of this check, and/or change the start time from Sunday early morning to the middle of the week.

    Does OMV provide any means of changing this? If not, could it be added to a future release?

  • * bump* Another resync getting in my way on a Sunday. I really, really would like to be able to change the resync schedule (without OMV changing it back).


    Not necessarily to reduce its frequency, but to move it to a better time, say, a weekday early morning.


    Please? *puppy eyes*

    • Offizieller Beitrag

    Another resync getting in my way on a Sunday. I really, really would like to be able to change the resync schedule (without OMV changing it back).


    Not necessarily to reduce its frequency, but to move it to a better time, say, a weekday early morning.


    Please?

    As far as I can tell, OMV doesn't maintain the /etc/cron.d/mdadm file. So, you should be able to change the time it runs.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Hmm, earlier you stated that OMV will revert any edits to that file.

    Bad assumption on my part.

  • Edited /etc/cron.d/mdadm and replaced 57 0 * * 0 with 56 23 * * 2 (Tuesday at 11:56 p.m.).


    Then issued

    Code
    echo 120000 > /proc/sys/dev/raid/speed_limit_max
    echo 40000 > /proc/sys/dev/raid/speed_limit_min

    to limit resync speeds to between 40MB/s and 120MB/s.


    Guess I'll find out how well it works on February 2nd. :)

  • And most users don't understand why they are using raid.... I don't use raid on my personal NAS at home but I do maintain hundreds of systems using raid at work. What does that tell you?

    Good question. I would answer it for the following way:

    Important data always needs backup. RAID is not a backup, but kind of risk management. It helps with two issues:

    • Backups are never up to date. So there will be always a chance of files going lost no matter what.
    • Restore from backup costs time.

    In case a disk dies and no raid is applied, both of the problems above kick in.


    But there is a third problem. Cost. The user decides if the cost of data loss is bigger than the cost for backup. In the area of less important data, raid can be a way to balance costs:

    • For less important data, raid is cheaper than full backup.

    So in short, raid balances cost and time.

    cpu Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
    omv 6.9.13-1 (Shaitan)

    kernel 6.1.0-0.deb11.11-amd64

  • I have changed /etc/cron.d/mdadm to "57 0 * 2,5,8,11 0". Lets see if that works :)

    cpu Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
    omv 6.9.13-1 (Shaitan)

    kernel 6.1.0-0.deb11.11-amd64

    • Offizieller Beitrag

    raid balances cost and time.

    That is neglecting a more common problem - accidental deletes and ransomware. Raid is actually worse with both of those and a delayed backup helps. mdadm without a CoW filesystem doesn't give you any bitrot detection either.

    raid is cheaper than full backup.

    How? If you have a two drive raid 1 array, you would have two disks. If you have two separate drives with scheduled rsync/borgbackup/rsnapshot, you have backup and the same cost. I guess if you have three drives that are really full where something like borgbackup can't compress and dedupe two drives to one, then you might need one more drive. One you have four or more drives, you probably aren't going to worry too much about an extra drive or two. Raid only gives you availability which most OMV uses can handle a little bit longer downtime.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!