degraded RAID but disk still there and works, raid doesn't rebuild

fredlepp · Feb 8th 2023

Hello. I’m coming to you because I’ve had a problem since yesterday.

a disk in my array is missing in the "raid management" tab, the raid is degraded, the volume is no longer mounted, but the disk is plugged in and visible in the list of drives and the smart test are green.

So I deleted the incriminated disk to make it appear in the list of available disks for the raid recovery, he accepted it, I turned up the volume. For now the raid is not rebuilt, but I have again access to my files under windows.

I don’t know what to do. yesterday I already did the same manipulation, but tonight, by making the server fun again, the same problem appears.

I just made the server reboot to see if it took into account the changes and if it launched the reconstruction of the raid, but it’s as if I did nothing, that is to say as explained above, more access to the RAID in windows, missing disk, degraded RAID but present and functional disk.

while waiting for an answer, I will try to run a deep smart test.

Thank you for reading me

geaves · Feb 9th 2023

I'm sorry trying to make sense of what you've written, but some information would be helpful, ssh into OMV and run the following two commands and post the output of each in a code box this -> </> symbol on thread bar, makes it easier to read

cat /proc/mdstat

blkid

fredlepp · Feb 9th 2023

Code

cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [ra id10]
md0 : active raid5 sdf[8](F) sda[0] sdg[6] sde[4] sdb[1] sdd[3] sdc[2] sdh[7]
13673676800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [8/7] [UUUUU _UU]
bitmap: 9/15 pages [36KB], 65536KB chunk
unused devices: <none>

Code

blkid
/dev/sda: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="b0103126-2c00-6bf3-9c56-8228e86f19ef" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdh: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="cd6b6e46-1d0e-a42c-0cb4-029a38fb04b8" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sde: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="991b1cc8-2460-096b-ba65-40b4249bae7f" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdg: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="a0abb8af-a0b7-7825-6be6-9de798db413d" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdd: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="0efbbe05-74f4-e707-bdca-683a3e47a3be" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdb: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="843dd1e1-ccbe-60bd-6465-edb50b046866" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdc: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="7c3d880c-36e5-1531-36b5-85ad221df929" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdi1: UUID="E889-8B4E" TYPE="vfat" PARTUUID="084782c2-9cab-4325-ae29-df1b9c58b97e"

/dev/sdi2: UUID="b96d8f61-99cb-42d3-8a95-bdb4a91fc4df" TYPE="ext4" PARTUUID="dc55364a-5131-4fc8-87f4-b95722624445"

/dev/sdi3: UUID="57de585b-b840-461d-bddc-2243b93ec5b5" TYPE="swap" PARTUUID="49944d3c-cde9-4fdf-a22a-bb2b09be2076"

/dev/md0: LABEL="Serveur" UUID="fdcf9e09-02e3-4ce7-a435-1a6a78a0a070" TYPE="ext4"

/dev/sdf: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="f8f3b374-190f-add3-c32c-f3372d247f00" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

root@openmediavault:~# cat /proc/mdstat

/dev/sdh: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="cd6b6e46-1d0e-a42c-0cb4-029a38fb04b8" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sde: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="991b1cc8-2460-096b-ba65-40b4249bae7f" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdg: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="a0abb8af-a0b7-7825-6be6-9de798db413d" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdd: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="0efbbe05-74f4-e707-bdca-683a3e47a3be" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdb: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="843dd1e1-ccbe-60bd-6465-edb50b046866" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdc: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="7c3d880c-36e5-1531-36b5-85ad221df929" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

/dev/sdi1: UUID="E889-8B4E" TYPE="vfat" PARTUUID="084782c2-9cab-4325-ae29-df1b9c58b97e"

/dev/sdi2: UUID="b96d8f61-99cb-42d3-8a95-bdb4a91fc4df" TYPE="ext4" PARTUUID="dc55364a-5131-4fc8-87f4-b95722624445"

/dev/sdi3: UUID="57de585b-b840-461d-bddc-2243b93ec5b5" TYPE="swap" PARTUUID="49944d3c-cde9-4fdf-a22a-bb2b09be2076"

/dev/md0: LABEL="Serveur" UUID="fdcf9e09-02e3-4ce7-a435-1a6a78a0a070" TYPE="ext4"

/dev/sdf: UUID="b609be46-06fb-3115-3497-57acf2fba372" UUID_SUB="f8f3b374-190f-add3-c32c-f3372d247f00" LABEL="openmediavault:Raid5" TYPE="linux_raid_member"

Display More

geaves · Feb 9th 2023

Ok according to cat /proc/mdstat the array is active and it looks as if one of those is marked as failed, post the output of;

mdadm --detail /dev/md0

So the array should display in Raid Management as clean/degraded

fredlepp · Feb 9th 2023

Code

login as: root
root@192.168.0.50's password:
Linux openmediavault 5.10.0-0.deb10.16-amd64 #1 SMP Debian 5.10.127-2~bpo10+1 (2022-07-28) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Feb  9 20:25:29 2023 from 192.168.0.20
root@openmediavault:~# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Mon Jul  4 19:11:31 2022
        Raid Level : raid5
        Array Size : 13673676800 (13040.23 GiB 14001.85 GB)
     Used Dev Size : 1953382400 (1862.89 GiB 2000.26 GB)
      Raid Devices : 8
     Total Devices : 8
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Thu Feb  9 22:10:30 2023
             State : clean, degraded
    Active Devices : 7
   Working Devices : 7
    Failed Devices : 1
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : bitmap

              Name : openmediavault:Raid5  (local to host openmediavault)
              UUID : b609be46:06fb3115:349757ac:f2fba372
            Events : 360344

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       8       32        2      active sync   /dev/sdc
       3       8       48        3      active sync   /dev/sdd
       4       8       64        4      active sync   /dev/sde
       -       0        0        5      removed
       6       8       96        6      active sync   /dev/sdg
       7       8      112        7      active sync   /dev/sdh

       8       8       80        -      faulty   /dev/sdf
root@openmediavault:~#

Display More

fredlepp · Feb 9th 2023

Quote from geaves

So the array should display in Raid Management as clean/degraded

it is display active/degraded

geaves · Feb 9th 2023

Quote from fredlepp

it is display active/degraded

OK, same thing, but the array is in a degraded state, the drive /dev/sdf has been removed by mdadm, hence the output from mdadm --detail

That drive needs replacing, if you need instructions on how to do that let me know.

But as a piece of advice, 8 drives in a raid 5 is not a good idea, once you get go over 4, raid 6 is a better option

fredlepp · Feb 9th 2023

ok, i will change the disk. i begin a backup to restrat with zfs raid solution but the place i need will be critical

It’s still weird that this record is already dead it must be barely 1 or 2 years old

geaves · Feb 9th 2023

Quote from fredlepp

It’s still weird that this record is already dead it must be barely 1 or 2 years old

Have you run a long SMART test on it? If it's only that old most drives have a 3 year warranty

fredlepp · Feb 9th 2023

long test passed, i will verify the bill, i have four of this hdd seagate model

geaves · Feb 9th 2023

Quote from fredlepp

long test passed

Passed is not a confirmation that the drive is not faulty you need to look at the output

fredlepp · Feb 11th 2023

the hard drive is still under warranty, RMA procedure triggered this morning. in addition I replace the disk with a new one, everything normally. thank's for helping.

degraded RAID but disk still there and works, raid doesn't rebuild

fredlepp Feb 11th 2023

fredlepp Feb 11th 2023

Participate now!