Raid 5 lässt sich nicht wieder herstellen nach Reboot

mdkder3 · Aug 15th 2022

Moin Zusammen,

ich habe seit Samstag leider das Problem das sich nach einigen Neustarts weil meine USV Probleme gemacht hat und jetzt bekomme ich das Raid 5 nicht wieder aktiv.

ich habe hier schon mal einige Sachen gezogen, versucht hier mal die Auszüge was ich so anliegt.

Code

root@hs-omv:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : inactive sdd[3](S) sdb[1](S) sda[0](S) sdc[2](S)
      31255582048 blocks super 1.2

Code

root@hs-omv:~# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
        Raid Level : raid0
     Total Devices : 4
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 4

              Name : hs-omv:Raid  (local to host hs-omv)
              UUID : 87860d3a:5a311365:5f35ba33:2015e4d7
            Events : 169033

    Number   Major   Minor   RaidDevice

       -       8       32        -        /dev/sdc
       -       8        0        -        /dev/sda
       -       8       48        -        /dev/sdd
       -       8       16        -        /dev/sdb

Display More

Ich weiß nicht warum hier Raid 1 steht das ist ein Raid5 und ich hoffe das bekommt man repariert

Code

root@hs-omv:~# cat /proc/partitions
major minor  #blocks  name

   8       32 7814026584 sdc
   8        0 7814026584 sda
   8       48 7814026584 sdd
   8       16 7814026584 sdb
   8       64  250059096 sde
   8       65  233447424 sde1
   8       66          1 sde2
   8       69   16609280 sde5
   7        0      63448 loop0
   7        1      44920 loop1
   7        2      44908 loop2
   7        3      63448 loop3
   7        4     116724 loop4
   7        5     116620 loop5

Display More

Code

root@hs-omv:~# mdadm --stop /dev/md127
mdadm: stopped /dev/md127

root@hs-omv:~# mdadm --assemble --force --verbose /dev/md127 /dev/sd[abcd]
mdadm: looking for devices for /dev/md127
mdadm: /dev/sda is identified as a member of /dev/md127, slot -1.
mdadm: /dev/sdb is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdc is identified as a member of /dev/md127, slot -1.
mdadm: /dev/sdd is identified as a member of /dev/md127, slot -1.
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: no uptodate device for slot 2 of /dev/md127
mdadm: no uptodate device for slot 3 of /dev/md127
mdadm: added /dev/sda to /dev/md127 as -1
mdadm: added /dev/sdc to /dev/md127 as -1
mdadm: added /dev/sdd to /dev/md127 as -1
mdadm: added /dev/sdb to /dev/md127 as 1
mdadm: /dev/md127 assembled from 0 drives, 1 rebuilding and 3 spares - not enough to start the array.

Display More

ich habe irgendwie die Befürchtung das es mit den Slots zu tun hat, kann es mir aber nicht erklären warum da "-1" steht
vllt kann mit jedem erklären wie ich das geändert bekomme und dadurch nicht alles weg ist.

Danke vorab

geaves · Aug 15th 2022

Not sure if I can help never seen anything like this before, either there is something corrupted or there is a hardware issue, as both point to the output of mdadm --detail.

That output is missing information in relation to the array, if you look at the output from this problem;

Code

Version : 1.2
  Creation Time : Sat Nov 11 17:45:46 2017
     Raid Level : raid1
     Array Size : 1953383512 (1862.89 GiB 2000.26 GB)
  Used Dev Size : 1953383512 (1862.89 GiB 2000.26 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent


  Intent Bitmap : Internal


    Update Time : Tue Jul 30 20:16:52 2019
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0


           Name : Zangs-NAS:Zangs
           UUID : 3ca82093:31bdaba6:e3fc2da7:8924c663
         Events : 14734


    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       1       8       16        1      active sync   /dev/sdb

Display More

you can compare it to yours.

Post the output of the following;

blkid

mdadm --examine /dev/sdb

mdadm --examine /dev/sda

mdkder3 · Aug 15th 2022

Quote from geaves
Not sure if I can help never seen anything like this before, either there is something corrupted or there is a hardware issue, as both point to the output of mdadm --detail.

That output is missing information in relation to the array, if you look at the output from this problem;
Code
Version : 1.2
  Creation Time : Sat Nov 11 17:45:46 2017
     Raid Level : raid1
     Array Size : 1953383512 (1862.89 GiB 2000.26 GB)
  Used Dev Size : 1953383512 (1862.89 GiB 2000.26 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent


  Intent Bitmap : Internal


    Update Time : Tue Jul 30 20:16:52 2019
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0


           Name : Zangs-NAS:Zangs
           UUID : 3ca82093:31bdaba6:e3fc2da7:8924c663
         Events : 14734


    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       1       8       16        1      active sync   /dev/sdb
Display More
you can compare it to yours.

Post the output of the following;

blkid
mdadm --examine /dev/sdb
mdadm --examine /dev/sda
Display More

Code

root@hs-omv:~# blkid
/dev/sdd: UUID="87860d3a-5a31-1365-5f35-ba332015e4d7" UUID_SUB="9f1ef59f-8e1c-4279-337c-86ce0dced58e" LABEL="hs-omv:Raid" TYPE="linux_raid_member"
/dev/sde1: UUID="2e26a2d0-5b69-4c62-9dfb-d4f4f3ada264" TYPE="ext4" PARTUUID="82468501-01"
/dev/sde5: UUID="e8b3ef0f-dba1-403d-bc63-e11cc5d73740" TYPE="swap" PARTUUID="82468501-05"
/dev/sda: UUID="87860d3a-5a31-1365-5f35-ba332015e4d7" UUID_SUB="783465ff-cf2c-b66c-ed30-c5f01ccb1c4a" LABEL="hs-omv:Raid" TYPE="linux_raid_member"
/dev/sdb: UUID="87860d3a-5a31-1365-5f35-ba332015e4d7" UUID_SUB="f76f0bbe-72d4-d716-639a-98ac88a9db5e" LABEL="hs-omv:Raid" TYPE="linux_raid_member"
/dev/sdc: UUID="87860d3a-5a31-1365-5f35-ba332015e4d7" UUID_SUB="24745d07-8959-1886-2b4c-9ed0c1cbf06b" LABEL="hs-omv:Raid" TYPE="linux_raid_member"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"

Display More

Code

root@hs-omv:~# mdadm --examine /dev/sda
/dev/sda:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8b
     Array UUID : 87860d3a:5a311365:5f35ba33:2015e4d7
           Name : hs-omv:Raid  (local to host hs-omv)
  Creation Time : Sat Feb 23 08:27:05 2019
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
     Array Size : 23441685504 (22355.73 GiB 24004.29 GB)
  Used Dev Size : 15627790336 (7451.91 GiB 8001.43 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
Recovery Offset : 0 sectors
   Unused Space : before=262056 sectors, after=688 sectors
          State : clean
    Device UUID : 783465ff:cf2cb66c:ed30c5f0:1ccb1c4a

Internal Bitmap : 8 sectors from superblock
    Update Time : Sun Aug 14 07:10:38 2022
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
       Checksum : c3284a93 - correct
         Events : 161518

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : .A.. ('A' == active, '.' == missing, 'R' == replacing)

Display More

Code

root@hs-omv:~# mdadm --examine /dev/sdb
/dev/sdb:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x9
     Array UUID : 87860d3a:5a311365:5f35ba33:2015e4d7
           Name : hs-omv:Raid  (local to host hs-omv)
  Creation Time : Sat Feb 23 08:27:05 2019
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
     Array Size : 23441685504 (22355.73 GiB 24004.29 GB)
  Used Dev Size : 15627790336 (7451.91 GiB 8001.43 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=688 sectors
          State : clean
    Device UUID : f76f0bbe:72d4d716:639a98ac:88a9db5e

Internal Bitmap : 8 sectors from superblock
    Update Time : Sun Aug 14 12:05:43 2022
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
       Checksum : fc7fd798 - correct
         Events : 169033

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : .... ('A' == active, '.' == missing, 'R' == replacing)

Display More

Bitte schön
ich hoffe das hilft weiter

geaves · Aug 15th 2022

OK;

mdadm --examine /dev/sdc

mdadm --examine /dev/sdd

This one I'm not sure about but;

cat /sys/block/md127/md/sync_action

mdkder3 · Aug 15th 2022

Quote from geaves

OK;

mdadm --examine /dev/sdc
mdadm --examine /dev/sdd

This one I'm not sure about but;

cat /sys/block/md127/md/sync_action

Display More

Code

root@hs-omv:~# mdadm --examine /dev/sdc
/dev/sdc:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x9
     Array UUID : 87860d3a:5a311365:5f35ba33:2015e4d7
           Name : hs-omv:Raid  (local to host hs-omv)
  Creation Time : Sat Feb 23 08:27:05 2019
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
     Array Size : 23441685504 (22355.73 GiB 24004.29 GB)
  Used Dev Size : 15627790336 (7451.91 GiB 8001.43 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=688 sectors
          State : clean
    Device UUID : 24745d07:89591886:2b4c9ed0:c1cbf06b

Internal Bitmap : 8 sectors from superblock
    Update Time : Sun Aug 14 12:05:43 2022
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
       Checksum : e62d34df - correct
         Events : 169033

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : .... ('A' == active, '.' == missing, 'R' == replacing)

Display More

Code

root@hs-omv:~# mdadm --examine /dev/sdd
/dev/sdd:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x9
     Array UUID : 87860d3a:5a311365:5f35ba33:2015e4d7
           Name : hs-omv:Raid  (local to host hs-omv)
  Creation Time : Sat Feb 23 08:27:05 2019
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 15627791024 (7451.91 GiB 8001.43 GB)
     Array Size : 23441685504 (22355.73 GiB 24004.29 GB)
  Used Dev Size : 15627790336 (7451.91 GiB 8001.43 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=688 sectors
          State : clean
    Device UUID : 9f1ef59f:8e1c4279:337c86ce:0dced58e

Internal Bitmap : 8 sectors from superblock
    Update Time : Sun Aug 14 12:05:43 2022
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
       Checksum : 92bbd4b5 - correct
         Events : 169033

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : .... ('A' == active, '.' == missing, 'R' == replacing)

Display More

Code

root@hs-omv:~# cat /sys/block/md127/md/sync_action
cat: /sys/block/md127/md/sync_action: Datei oder Verzeichnis nicht gefunden

mdkder3 · Aug 15th 2022

was wäre wenn ich die Platten in ein frisches System einbaue
Könnte ich dann ein rebuild machen??

geaves · Aug 15th 2022

Right 3 of the drives are registering as spares so the array is never going to rebuild, I was sceptical about the /sys/block command as this is an Ubuntu option but work a shot.

One more output

cat /etc/mdadm/mdadm.conf

geaves · Aug 15th 2022

Quote from mdkder3

what if i put the disks in a fresh system
could i do a rebuild then

If you redeployed then you would do it without data drives connected, once the system was up running and working then connect the drives

mdkder3 · Aug 15th 2022

Quote from geaves

Right 3 of the drives are registering as spares so the array is never going to rebuild, I was sceptical about the /sys/block command as this is an Ubuntu option but work a shot.

One more output

cat /etc/mdadm/mdadm.conf

Code

oot@hs-omv:~# cat /etc/mdadm/mdadm.conf
# This file is auto-generated by openmediavault (https://www.openmediavault.org)
# WARNING: Do not edit this file, your changes will get lost.

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR mdkder31@googlemail.com
MAILFROM root

# definitions of existing MD arrays
ARRAY /dev/md/Raid metadata=1.2 name=hs-omv:Raid UUID=87860d3a:5a311365:5f35ba33:2015e4d7

Display More

mdkder3 · Aug 15th 2022

Quote from geaves

If you redeployed then you would do it without data drives connected, once the system was up running and working then connect the drives

Okay also System aufsetzten
System ausschalten
die HDDs anschließen und dann mit welchen Befehl genau das Rebuild starten??

hiermit?
mdadm --assemble --force --verbose /dev/md127 /dev/sd[abcd]

geaves · Aug 15th 2022

If you installed OMV6 on a spare drive or usb flash drive (OMV5 and Debian 10 is EOL) once the system is up and running updates installed, date and time set etc, shut down connect your drives and during the reboot process OMV should detect the raid signatures and present the array to OMV.

Another option might be to try systsemrescuecd in omc extras, this installs the software and should allow for the array to be assembled from the cli, exit out and reboot should bring the array back to omv. This is just a possibility, your error is quite unique, one that I have never seen nor experienced before.

mdkder3 · Aug 15th 2022

Quote from geaves

If you installed OMV6 on a spare drive or usb flash drive (OMV5 and Debian 10 is EOL) once the system is up and running updates installed, date and time set etc, shut down connect your drives and during the reboot process OMV should detect the raid signatures and present the array to OMV.

Another option might be to try systsemrescuecd in omc extras, this installs the software and should allow for the array to be assembled from the cli, exit out and reboot should bring the array back to omv. This is just a possibility, your error is quite unique, one that I have never seen nor experienced before.

Okay mit der systsemrescuecd komm ich irgendwie nicht zurecht

ich werde jetzt mal ein zweites System fertig machen und das mit dem Umbau testen
ich werde bescheid geben was drauß geworden ist.

geaves · Aug 15th 2022

I have just run up an OMV6 VM created a raid5 from three virtual disks and ran the cat /sys/block that I requested you run, this is the output;

Code

root@openmediavault6:~# cat /sys/block/md0/md/sync_actionidle

the above is what I would expect as the system is doing nothing and the command works!! this would also show output if this was OMV5

Yours returned -> file or directory not found, to confirm the raid reference has not changed you would have check cat /proc/mdstat, if it's still md127 then there could be some corruption. I am assuming at this point your array was created using the GUI

mdkder3 · Aug 15th 2022

ja genau, das Raid wurde über die GUI erstellt

und ja Ausgabe kommt immer noch "Datei oder Verzeichnis nicht gefunden"

mdkder3 · Aug 15th 2022

Code

root@server:~# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
        Raid Level : raid0
     Total Devices : 3
       Persistence : Superblock is persistent

             State : inactive
   Working Devices : 3

              Name : hs-omv:Raid
              UUID : 87860d3a:5a311365:5f35ba33:2015e4d7
            Events : 161518

    Number   Major   Minor   RaidDevice

       -       8       32        -        /dev/sdc
       -       8       48        -        /dev/sdd
       -       8       16        -        /dev/sdb
root@server:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : inactive sdb[3](S) sdd[2](S) sdc[1](S)
      23441686536 blocks super 1.2

unused devices: <none>
root@server:~#

Display More

das bekomme ich als anzeige im neuen System
Er erkennt das auch nur als Raid0 nicht als Raid 5

geaves · Aug 15th 2022

Quote from mdkder3

He only recognizes it as Raid0 not as Raid 5

That's because it's inactive and you have the same issue with that mdadm --detail as your first post

mdkder3 · Aug 15th 2022

Code

root@hs-omv:~# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Mon Aug 15 13:40:16 2022
        Raid Level : raid5
        Array Size : 23441683392 (22355.73 GiB 24004.28 GB)
     Used Dev Size : 7813894464 (7451.91 GiB 8001.43 GB)
      Raid Devices : 4
     Total Devices : 4
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Mon Aug 15 13:40:17 2022
             State : clean
    Active Devices : 4
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 64K

Consistency Policy : bitmap

              Name : server:127
              UUID : 1c85c925:03bd7d21:3136fea7:e700b544
            Events : 2

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       48        1      active sync   /dev/sdd
       2       8       16        2      active sync   /dev/sdb
       3       8       32        3      active sync   /dev/sdc

Display More

also das habe ich schon mal wieder geschafft, durch das zweit System.
Platten hängen jetzt wieder im Original System
jetzt muss nur noch der Name und die UUID wieder passen, das ich hoffentlich wieder alles zu sehen bekomme.

Frage: wie geht das?

geaves · Aug 15th 2022

Quote from mdkder3

Question: how does it work

I'm sorry I am at a total loss as to what is happening or why, whilst that array appears to be intact if the array is mounted then the share on the array should still be there

mdkder3 · Aug 15th 2022

Quote from geaves

I'm sorry I am at a total loss as to what is happening or why, whilst that array appears to be intact if the array is mounted then the share on the array should still be there

alles gut
du hast mir ja schon ein bisschen weitergeholfen

Raid 5 lässt sich nicht wieder herstellen nach Reboot

mdkder3 Aug 15th 2022

Participate now!

Tags