Raid5 verschwindet nach jedem Reboot

Timothee · Mar 30th 2016

Hallo Leute,
zunächst einmal ein großen Dank an alle, die an OMV beteiligt. Es ist klasse, was ihr auf die Beine gestellt hab.
Ich habe das Problem, dass nach jedem Reboot eins meiner zwei Raid5-Verbunde verschwindet. Ich kann dann mittels folgendem Befehl das verschwundene Raidverbund wieder einbinden.

Code

mdadm --create /dev/md/Pool3xHDD --chunk=512 --raid-devices=3 --spare-devices=0 --assume-clean --level=5 --name=homeserver:Pool3xHDD /dev/sdf /dev/sdg /dev/sdh --verbose

Alle Daten sind danach wieder vorhanden und scheinen in Ordnung. Diesen Workaround hab ich durch einige Stunden Recherche gefunden . Ich dachte schon, ich kann mein Raidverbund auflösen und neu initialisieren.
Jedoch kann das nicht als Dauerlösung eingesetzt werden, da der Server eigentlich nur bei Bedarf per WOL gestartet werden soll.

Hier mal die Ausgaben nach einem Neustart für die aufgeführten Befehle im angepinnten Thema "Degraded or missing raid array questions".

Display Spoiler

Code

cat /proc/mdstat:
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[0] sde[4] sdd[3] sdc[2] sdb[1]
      15627548672 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
      bitmap: 0/30 pages [0KB], 65536KB chunk


unused devices: <none>

Code

blkid:
/dev/sdc: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="b0f2ce16-be35-3d8b-2ea6-a4f4327eae71" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdb: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="85056897-6b2c-a35f-28f5-34196c9078b4" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdi1: UUID="4c11b28c-7d9a-4a9f-8d53-e1b4e91492c8" TYPE="ext4" PARTUUID="6390d39f-01"
/dev/sdi5: UUID="2fb98d7d-fd00-49cd-b9c7-890452a79fc2" TYPE="swap" PARTUUID="6390d39f-05"
/dev/sde: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="514e23dc-7f24-d7ef-2a73-658be6b7ed16" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sda: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="0266268f-eab0-97a8-3371-061d463906e2" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdf: UUID="8e6903ce-5c70-f699-3588-2f665846e42d" UUID_SUB="9a3b29f1-bd4a-4316-2500-c0e58994f524" LABEL="homeserver:Pool3xHDD" TYPE="linux_raid_member"
/dev/md0: LABEL="Raid5xHDD" UUID="e36d1e07-3b7d-4048-b790-959e0f345579" TYPE="xfs"
/dev/sdd: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="65eac07d-67a7-4cbb-78a4-5991f2f82fa1" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdh1: PARTLABEL="Basic data partition" PARTUUID="589425a8-d5a6-4c9c-a929-733d9409bd95"
/dev/sdg1: PARTLABEL="Basic data partition" PARTUUID="91e5f2a0-d504-42dd-8ff4-676afed98c21"

Display More

Code

fdisk -l:
Disk /dev/sdc: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdh: 7,3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: DD66ED13-30AF-4848-A2C0-DB79209451F0


Device     Start         End     Sectors  Size Type
/dev/sdh1   2048 15628052479 15628050432  7,3T Microsoft basic data


Disk /dev/sdi: 119,2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x6390d39f


Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdi1  *         2048 239855615 239853568 114,4G 83 Linux
/dev/sdi2       239857662 250068991  10211330   4,9G  5 Extended
/dev/sdi5       239857664 250068991  10211328   4,9G 82 Linux swap / Solaris


Disk /dev/sde: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sda: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdg: 7,3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: C3E7FD71-6072-4C1E-BE1A-833C3751BAB1


Device     Start         End     Sectors  Size Type
/dev/sdg1   2048 15628052479 15628050432  7,3T Microsoft basic data


Disk /dev/sdf: 7,3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdd: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/md0: 14,6 TiB, 16002609840128 bytes, 31255097344 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 2097152 bytes

Display More

Code

cat /etc/mdadm/mdadm.conf:
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#


# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions


# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes


# automatically tag new arrays as belonging to the local system
HOMEHOST <system>


# definitions of existing MD arrays
ARRAY /dev/md0 metadata=1.2 name=homeserver:Pool5xHDD UUID=389bf470:2ad36f28:7318ecf6:27e9fabb
ARRAY /dev/md/Pool3xHDD metadata=1.2 name=homeserver:Pool3xHDD UUID=8e6903ce:5c70f699:35882f66:5846e42d


# instruct the monitoring daemon where to send mail alerts
MAILADDR timXXX@XXX.de

Display More

Code

mdadm --detail --scan --verbose:
ARRAY /dev/md0 level=raid5 num-devices=5 metadata=1.2 name=homeserver:Pool5xHDD UUID=389bf470:2ad36f28:7318ecf6:27e9fabb
   devices=/dev/sda,/dev/sdb,/dev/sdc,/dev/sdd,/dev/sde

Hier jetzt die Ausgaben beim Wiedereinbinden / Erstellen des verschwundenen Raidverbunds:

Display Spoiler

Ich hab leider keine passende Lösung für mich gefunden, deshalb wende ich mich jetzt an euch. Ich hoffe ihr könnt mir helfen. Denn das ganze als Autostartskript zu automatisieren ist für mich nicht der richtige Weg.

Beste Grüße
Timothee

Timothee · Mar 30th 2016

Da man nicht mehr als 10000 Zeichen posten kann, hier noch die Ausgaben nach dem Wiedereinbinden / Erstellen:

Display Spoiler

Code

cat /proc/mdstat:
Personalities : [raid6] [raid5] [raid4]
md127 : active raid5 sdh[2] sdg[1] sdf[0]
      15627790336 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
      bitmap: 0/59 pages [0KB], 65536KB chunk


md0 : active raid5 sda[0] sde[4] sdd[3] sdc[2] sdb[1]
      15627548672 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
      bitmap: 0/30 pages [0KB], 65536KB chunk


unused devices: <none>

Display More

Code

blkid:
/dev/sdc: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="b0f2ce16-be35-3d8b-2ea6-a4f4327eae71" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdb: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="85056897-6b2c-a35f-28f5-34196c9078b4" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdi1: UUID="4c11b28c-7d9a-4a9f-8d53-e1b4e91492c8" TYPE="ext4" PARTUUID="6390d39f-01"
/dev/sdi5: UUID="2fb98d7d-fd00-49cd-b9c7-890452a79fc2" TYPE="swap" PARTUUID="6390d39f-05"
/dev/sde: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="514e23dc-7f24-d7ef-2a73-658be6b7ed16" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sda: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="0266268f-eab0-97a8-3371-061d463906e2" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdf: UUID="e5f569b7-7000-898e-8616-4e0094dce241" UUID_SUB="a2848144-f5de-3aa6-ad5b-64093fe5d1c8" LABEL="homeserver:Pool3xHDD" TYPE="linux_raid_member"
/dev/md0: LABEL="Raid5xHDD" UUID="e36d1e07-3b7d-4048-b790-959e0f345579" TYPE="xfs"
/dev/sdd: UUID="389bf470-2ad3-6f28-7318-ecf627e9fabb" UUID_SUB="65eac07d-67a7-4cbb-78a4-5991f2f82fa1" LABEL="homeserver:Pool5xHDD" TYPE="linux_raid_member"
/dev/sdh: UUID="e5f569b7-7000-898e-8616-4e0094dce241" UUID_SUB="e204c0c9-8039-c096-2712-0f9490429bf5" LABEL="homeserver:Pool3xHDD" TYPE="linux_raid_member"
/dev/sdg: UUID="e5f569b7-7000-898e-8616-4e0094dce241" UUID_SUB="92b50755-a000-b12e-5ed2-8ecbcac680e1" LABEL="homeserver:Pool3xHDD" TYPE="linux_raid_member"
/dev/md127: LABEL="Raid3xHDD" UUID="889a45fd-7f7b-44c3-a3ae-056f3e9945c8" TYPE="xfs"

Display More

Code

fdisk -l:


Disk /dev/sdc: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
The primary GPT table is corrupt, but the backup appears OK, so that will be used.


Disk /dev/sdh: 7,3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: DD66ED13-30AF-4848-A2C0-DB79209451F0


Device     Start         End     Sectors  Size Type
/dev/sdh1   2048 15628052479 15628050432  7,3T Microsoft basic data


Disk /dev/sdi: 119,2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x6390d39f


Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdi1  *         2048 239855615 239853568 114,4G 83 Linux
/dev/sdi2       239857662 250068991  10211330   4,9G  5 Extended
/dev/sdi5       239857664 250068991  10211328   4,9G 82 Linux swap / Solaris


Disk /dev/sde: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sda: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
The primary GPT table is corrupt, but the backup appears OK, so that will be used.


Disk /dev/sdg: 7,3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: C3E7FD71-6072-4C1E-BE1A-833C3751BAB1


Device     Start         End     Sectors  Size Type
/dev/sdg1   2048 15628052479 15628050432  7,3T Microsoft basic data


Disk /dev/sdf: 7,3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdd: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/md0: 14,6 TiB, 16002609840128 bytes, 31255097344 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 2097152 bytes


Disk /dev/md127: 14,6 TiB, 16002857304064 bytes, 31255580672 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes

Display More

Code

cat /etc/mdadm/mdadm.conf:
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#


# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions


# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes


# automatically tag new arrays as belonging to the local system
HOMEHOST <system>


# definitions of existing MD arrays
ARRAY /dev/md0 metadata=1.2 name=homeserver:Pool5xHDD UUID=389bf470:2ad36f28:7318ecf6:27e9fabb
ARRAY /dev/md/Pool3xHDD metadata=1.2 name=homeserver:Pool3xHDD UUID=8e6903ce:5c70f699:35882f66:5846e42d


# instruct the monitoring daemon where to send mail alerts
MAILADDR timXXX@XXX.de
MAILFROM root

Display More

Code

mdadm --detail --scan --verbose:
ARRAY /dev/md0 level=raid5 num-devices=5 metadata=1.2 name=homeserver:Pool5xHDD UUID=389bf470:2ad36f28:7318ecf6:27e9fabb
   devices=/dev/sda,/dev/sdb,/dev/sdc,/dev/sdd,/dev/sde
ARRAY /dev/md/Pool3xHDD level=raid5 num-devices=3 metadata=1.2 name=homeserver:Pool3xHDD UUID=e5f569b7:7000898e:86164e00:94dce241
   devices=/dev/sdf,/dev/sdg,/dev/sdh

ryecoaaron · Mar 30th 2016

Some of the post didn't translate well but try running update-initramfs after recreating the array. Then reboot to see if it disappears.

While you seem to be lucky, using the create flag with data on the array is dangerous. I guess the assume-clean flag is saving the data.

Timothee · Mar 30th 2016

Thanks for the fast reply. Sorry for my bad english. This is the reason why i wrote the post in german ^^.
I will try it when i come home. Should i execute update-initramfs without parameters? I am a relativily newbie in linux / linux raid

Best regards
Timothee

ryecoaaron · Mar 30th 2016

I would try update-initramfs -u all

Timothee · Mar 30th 2016

I tried it, but it aborted because of "extra argument all".
Then i executed it without "all" and it generated the file.
But after a reboot the same situation - my "Raid3xHDD" Raid5 is missing.

ryecoaaron · Mar 30th 2016

Are there any usb drives in the array?

Timothee · Mar 30th 2016

nope, all drives are connected with sata.

ryecoaaron · Mar 30th 2016

All I can think is that a drive is starting up slow or starting to fail??

Timothee · Mar 30th 2016

When my server booting i guess i see a message which indicates that a drive is not available. Tomorrow i look again and then i post the message here.
When that is the problem, is it solvable?

Best regards
Timothee

ryecoaaron · Mar 30th 2016

I would have to see the message first.

Teuflor · Apr 3rd 2016

Ich hab leider das gleiche Problem. ein Raid 5 aus 4 Platten, nach dem Neustart ist es nicht mehr vorhaden.

Wird aber noch gefunden:

Code

mdadm --detail --scan --verbose
ARRAY /dev/md0 level=raid5 num-devices=4 metadata=1.2 name=openmediavault:RAID UUID=51e8ea9a:9add00f3:8d3dc15c:b33080ae                                                  
   devices=/dev/sda,/dev/sdd

Naja, so halbwegs. Das Raid ist eigentlich aus sda, sdb, sdc und sdd aufgebaut...
Mit dem Befehl mit dem du dein Raid wieder zum laufen bekommst, wirkt bei mir leider nicht.

Teuflor · Apr 3rd 2016

Code

root@openmediavault:~# mdadm --create /dev/md/RAID --chunk=512 --raid-devices=4 --spare-devices=0 --assume-clean --level=5 --name=openmediavault:RAID /dev/sda /dev/sdb /
dev/sdc /dev/sdd --verbose                                                                                                                                               
mdadm: layout defaults to left-symmetric                                                                                                                                 
mdadm: layout defaults to left-symmetric                                                                                                                                 
mdadm: super1.x cannot open /dev/sda: Device or resource busy                                                                                                            
mdadm: failed container membership check                                                                                                                                 
mdadm: cannot open /dev/sda: Device or resource busy

funktioniert leider bei mir nicht, äußerst ärgerlich bei einem frischen Server.

Participate now!