Help! Me too... Raid5 disappeared after Update to OMV 3.0.85 and reboot

CryptoWorX · 25. Juli 2017

Hi everyone,

got a big problem and got stuck in finding a solution...

Last week I installed OMV and created a Raid 5 consisting of 5 3TB WD Red Drives through the WebGUI (/dev/sda - sde). After initialization of the Array I created a filesystem (EXT4), started to set up my shares etc. and meanwhile copied over 6 TB Data on the Array (which are very important to me...). Today I saw in the WebGUI some Updates, incl. OMV 3.0.85. Installed them and thought a reboot wouldn't be a bad idea. Indeed, it was a very bad idea because after the reboot my Raid 5 is gone...

I've already read several threads here and in general linux forums which are concernd with that kind of problem, but none of the solutions worked for me. So I decided to ask for assistance here.

Code

root@saturn:~# cat /proc/mdstat
Personalities :
unused devices: <none>


root@saturn:~# blkid
/dev/sdf1: UUID="03879aad-ba99-4f69-8833-45131ed96a23" TYPE="ext4" PARTUUID="27b05a98-01"
/dev/sdf5: UUID="df39d7cc-ae0c-44eb-a364-bdd4bdb983f9" TYPE="swap" PARTUUID="27b05a98-05"
/dev/sdb: PTUUID="31f67c34-6aec-11e7-8096-a0369f3afccf" PTTYPE="gpt"
/dev/sdd: PTUUID="335fea84-6aec-11e7-8096-a0369f3afccf" PTTYPE="gpt"
/dev/sde: PTUUID="34172b6f-6aec-11e7-8096-a0369f3afccf" PTTYPE="gpt"
/dev/sdc: PTUUID="32aae6e3-6aec-11e7-8096-a0369f3afccf" PTTYPE="gpt"
/dev/sda: PTUUID="313bfa94-6aec-11e7-8096-a0369f3afccf" PTTYPE="gpt"


root@saturn:~# fdisk -l | grep "Disk "
Partition 3 does not start on physical sector boundary.
Disk /dev/sdb: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk identifier: 31F67C34-6AEC-11E7-8096-A0369F3AFCCF
Disk /dev/sdd: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk identifier: 335FEA84-6AEC-11E7-8096-A0369F3AFCCF
Disk /dev/sde: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk identifier: 34172B6F-6AEC-11E7-8096-A0369F3AFCCF
Disk /dev/sdc: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk identifier: 32AAE6E3-6AEC-11E7-8096-A0369F3AFCCF
Disk /dev/sda: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk identifier: 313BFA94-6AEC-11E7-8096-A0369F3AFCCF
Disk /dev/sdf: 232,9 GiB, 250059350016 bytes, 488397168 sectors
Disk identifier: 0x27b05a98


root@saturn:~# cat /etc/mdadm/mdadm.conf
# mdadm.conf
DEVICE partitions
CREATE owner=root group=disk mode=0660 auto=yes
HOMEHOST <system>
ARRAY /dev/md0 metadata=1.2 name=saturn:DATA UUID=56fb3bfe:8fd4d46b:f5c0a165:65e8b277


root@saturn:~# mdadm --detail --scan --verbose
root@saturn:~#

Alles anzeigen

After reading several threads I tried:

Code

root@saturn:~# mdadm --examine /dev/sda
/dev/sda:
   MBR Magic : aa55
Partition[0] :   4294967295 sectors at            1 (type ee)


root@saturn:~# mdadm --examine /dev/sdb
/dev/sdb:
   MBR Magic : aa55
Partition[0] :   4294967295 sectors at            1 (type ee)


root@saturn:~# mdadm --examine /dev/sdc
/dev/sdc:
   MBR Magic : aa55
Partition[0] :   4294967295 sectors at            1 (type ee)


root@saturn:~# mdadm --examine /dev/sdd
/dev/sdd:
   MBR Magic : aa55
Partition[0] :   4294967295 sectors at            1 (type ee)


root@saturn:~# mdadm --examine /dev/sde
/dev/sde:
   MBR Magic : aa55
Partition[0] :   4294967295 sectors at            1 (type ee)


root@saturn:~# mdadm --assemble --force --verbose /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: looking for devices for /dev/md0
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: /dev/sda has no superblock - assembly aborted
root@saturn:~#


root@saturn:~# mdadm --assemble --scan -v
mdadm: looking for devices for /dev/md0
mdadm: no RAID superblock on /dev/sdf5
mdadm: no RAID superblock on /dev/sdf2
mdadm: no RAID superblock on /dev/sdf1
mdadm: no RAID superblock on /dev/sdf
mdadm: no RAID superblock on /dev/sda
mdadm: no RAID superblock on /dev/sdc
mdadm: no RAID superblock on /dev/sde
mdadm: no RAID superblock on /dev/sdd
mdadm: no RAID superblock on /dev/sdb

Alles anzeigen

Any help would be highly appreciated

Thanks in advance & kind regards

Chris (aka. CryptoWorX)

tkaiser · 25. Juli 2017

Zitat von CryptoWorX

thought a reboot wouldn't be a bad idea. Indeed, it was a very bad idea because after the reboot my Raid 5 is gone...

Did you reboot between RAID initialization and this last reboot already?

CryptoWorX · 25. Juli 2017

Zitat von tkaiser

Did you reboot between RAID initialization and this last reboot already?

I had to look through the logs - No...

As there are obviously no partitions listed/created on the Array's Members - could it have to do with "DEVICE partitions" in the mdadm.conf?

CryptoWorX · 25. Juli 2017

Another thought...

I checked the Disks with Testdisk and found recoverable partitions:

Code

TestDisk 6.14, Data Recovery Utility, July 2013
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org


Disk /dev/sda - 3000 GB / 2794 GiB - CHS 364801 255 63
     Partition               Start        End    Size in sectors
>  Linux RAID               0  32 33 364785  17 32 5860270080 [saturn:0]


Disk /dev/sdb - 3000 GB / 2794 GiB - CHS 364801 255 63
     Partition               Start        End    Size in sectors
>  Linux RAID               0  32 33 364785  17 32 5860270080 [saturn:0]


Disk /dev/sdc - 3000 GB / 2794 GiB - CHS 364801 255 63
     Partition               Start        End    Size in sectors
>  Linux RAID               0  32 33 364785  17 32 5860270080 [saturn:0]


Disk /dev/sdd - 3000 GB / 2794 GiB - CHS 364801 255 63
     Partition               Start        End    Size in sectors
>  Linux RAID               0  32 33 364785  17 32 5860270080 [saturn:0]


Disk /dev/sde - 3000 GB / 2794 GiB - CHS 364801 255 63
     Partition               Start        End    Size in sectors
>  Linux RAID               0  32 33 364785  17 32 5860270080 [saturn:0]

Alles anzeigen

Maybe partitions were not written to Disks? But how could the Array then have been accessible before????

I'm confused...

tkaiser · 25. Juli 2017

Zitat von CryptoWorX

As there are obviously no partitions listed/created on the Array's Members - could it have to do with "DEVICE partitions" in the mdadm.conf?

No idea. I deal now for 2 decades with (failed) RAIDs and that's the reason I avoid RAID whereever possible. As you might've already noticed RAID is only about availability but provides zero data protection features (and when you trust in it without extensive testing you're always lost). I hope you've a backup of your data?

CryptoWorX · 25. Juli 2017

Zitat von tkaiser

I hope you've a backup of your data?

Not of all - the OMV NAS ist mainly used as backup-device for my Win2012R2 Server. Some shares are only synchronized twice a week, so I might loose some data... Not essentially, but annoying...

What Do you think on the Testdisk results? Worth to try?

tkaiser · 25. Juli 2017

Zitat von CryptoWorX

What Do you think on the Testdisk results? Worth to try?

No idea. I dealt the last time with mdraid and RAID5 maybe 18 years ago and gave up entirely on the idea (especially data recovery since prevention + backup is the simpler approach). But since I also read some threads here the last few weeks about RAIDs vanishing I would believe it's related to the config not written to disk (so everything is gone after the first reboot -- better do this the next time before filling the array with data).

Hope you succeed reconstructing the array and then take your time to come up with better concepts (RAID without any backup is nothing I would even think about)

cabrio_leo · 25. Juli 2017

Zitat von tkaiser

I would believe it's related to the config not written to disk

But if you think it´s not a user error but a problem of OMV or the underlying Linux, shouldn´t it further explored?
Should @CryptoWorX file a bug report?

CryptoWorX · 25. Juli 2017

Zitat von tkaiser

I would believe it's related to the config not written to disk (so everything is gone after the first reboot -- better do this the next time before filling the array with data)

Yes, in my opinion too... But why? Created the Array, clicked "Save changes". After initializing the Array, created the filesystem, clicked "Save changes" and so on.

Will try what happens, when I recover the partitions with Testdisk. Can't get even more worse at the moment...

ryecoaaron · 25. Juli 2017

Zitat von cabrio_leo

Should @CryptoWorX file a bug report?

No. I've tried for a long time to figure out why people lose their arrays on reboot. Never found a consistent reason (other than consumer hardware or not wiping the drive well or usb drives or on an RPi). I have used and still use mdadm raid and have never had a problem. I don't think a bug report would help.

Zitat von CryptoWorX

What Do you think on the Testdisk results? Worth to try?

OMV doesn't use partitions for mdadm raid. If you restored partitions that were leftover from previous use of the drive, it would probably ruin the array. I would boot systemrescuecd on the box to get a newer set of mdadm tools and try to force assemble the array again.

ryecoaaron · 25. Juli 2017

Zitat von CryptoWorX

Can't get even more worse at the moment.

Yes it can....

johnw · 26. Juli 2017

There is a decent man page on mdadm. Lots of options and some may be of interest - maybe assemble but there are others. Some googling on specific options might help further.

https://linux.die.net/man/8/mdadm

3.4 seems to be the latest stable release.

I've been using mdadm for mirrored raid on my pc for some time now. 4 and a bit years but only recently updated. Curiously when I installed the full system update it ignored the fact that I mounted md0 to /home. I have wondered if this was a compatibility problem however it turned out that one of the disks had been failed. I suspect that was down to a single solitary power fail. I don't check the array that often so can't be 100% sure.

Seems to me that many linux disk formats are great at spotting problems and the better they are at that the worse they are at avoiding them. I'm inclined to say all actually. Maybe ext2 gets writes done as quickly as possible.

Edit
Looking at the help built into mdadm there are lots of options. Far more than there used to be. Subtopics are obtained by following the option with --help. It's pretty complicated.

John
-

Jetzt mitmachen!