Raid missing after upgrade from OMV 4 to 5

NeXTguy · Jul 24th 2020

Last night I upgraded OMV from version 4.x to 5.5.4-1 using the instructions and script from this post by ryecoaaron: RE: OMV 5.0 - finally out! :-)

The upgrade went smoothly but afterwards my raid is missing in action. The three drives that should make up the array are there, but the raid isn't assembled, and nothing is showing in Storage - RAID in the web interface. I've found the FAQ on this topic. It didn't help me resolve the issue. So, I'm including the requested information in the hope someone has an idea here.

mdstat

Code

root@mimas:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : inactive sdc[1](S) sdd[2](S)
3906767024 blocks super 1.2
unused devices: <none>

blkid

Code

root@mimas:~# blkid
/dev/sda1: UUID="93472a57-7231-44f9-82ca-5257364bcd42" TYPE="ext4" PARTUUID="b8e2fdeb-01"
/dev/sda3: LABEL="data0" UUID="2c5026ca-d497-4dc5-b570-251723e1e8b7" TYPE="ext4" PARTUUID="b8e2fdeb-03"
/dev/sda5: UUID="dba378a9-ae33-4486-935e-bf403b814154" TYPE="swap" PARTUUID="b8e2fdeb-05"
/dev/sdc: UUID="dafdd650-f6ff-f4c0-aea7-f00edd3676ed" UUID_SUB="a325b8d7-3270-9fe7-e298-29f31c10f0d8" LABEL="mimas:mainpool" TYPE="linux_raid_member"
/dev/sdd: UUID="dafdd650-f6ff-f4c0-aea7-f00edd3676ed" UUID_SUB="96d76f24-0e6e-3ea5-7085-2a46f59d6780" LABEL="mimas:mainpool" TYPE="linux_raid_member"
/dev/sdb1: PARTLABEL="primary" PARTUUID="64e56ceb-db8a-4e32-81ac-a7648a7df2a3"

fdisk

Code

root@mimas:~# fdisk -l | grep "Disk "
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WDC WD20EZRX-00D
Disk identifier: 13660068-638B-438E-999D-317819B1B4DB
Disk /dev/sda: 59.6 GiB, 64023257088 bytes, 125045424 sectors
Disk model: M4-CT064M4SSD2  
Disk identifier: 0xb8e2fdeb
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WDC WD20EARS-00M
Disk /dev/sdd: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WDC WD20EZRX-00D

Display More

mdadm

Code

root@mimas:~# cat /etc/mdadm/mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# definitions of existing MD arrays
ARRAY /dev/md/mimas:mainpool metadata=1.2 name=mimas:mainpool UUID=dafdd650:f6fff4c0:aea7f00e:dd3676ed

# instruct the monitoring daemon where to send mail alerts
MAILADDR XXXXXXXXXXXXXXXX
MAILFROM root

Display More

Code

root@mimas:~# mdadm --detail --scan --verbose
INACTIVE-ARRAY /dev/md127 num-devices=2 metadata=1.2 name=mimas:mainpool UUID=dafdd650:f6fff4c0:aea7f00e:dd3676ed
   devices=/dev/sdc,/dev/sdd

You can see four drives in the machine, a 60GB partitioned SSD for system and data (sda) and the three WD 2TB drives for the RAID (sdb/c/d). Drives are there, it worked before the upgrade, but now nothing. Trying to assemble the raid from the command line doesn't do anything, command just returns, no error message, no result.

Code

root@mimas:~# mdadm --assemble --scan
root@mimas:~#

I'm reasonably competent in Linux administration but not an expert on raids. Am I missing something obvious? Any help appreciated.

geaves · Jul 24th 2020

Quote from NeXTguy

missing in action

Search party reporting, try these coordinates mdadm --stop /dev/md127 -> mdadm --assemble --force --verbose /dev/md127 /dev/sd[cd]

NeXTguy · Jul 24th 2020

Thank you, this helps a bit. At least I can get at the data. But it doesn't give me back a fully functioning RAID:

Code

root@mimas:~# mdadm --stop /dev/md127
mdadm: stopped /dev/md127
root@mimas:~# mdadm --assemble --force --verbose /dev/md127 /dev/sd[cd]
mdadm: looking for devices for /dev/md127
mdadm: /dev/sdc is identified as a member of /dev/md127, slot 1.
mdadm: /dev/sdd is identified as a member of /dev/md127, slot 2.
mdadm: no uptodate device for slot 0 of /dev/md127
mdadm: added /dev/sdd to /dev/md127 as 2
mdadm: added /dev/sdc to /dev/md127 as 1
mdadm: /dev/md127 has been started with 2 drives (out of 3).

It is missing sdb. Did you leave that off on purpose? Because it's reported differently? And if so, any idea why sdb must be treated differently?

geaves · Jul 24th 2020

Quote from NeXTguy

It is missing sdb. Did you leave that off on purpose

No, that's because it's not listed in blkid as a linux raid member

Quote from NeXTguy

Because it's reported differently

sdb reports this in blkid /dev/sdb1: PARTLABEL="primary" PARTUUID="64e56ceb-db8a-4e32-81ac-a7648a7df2a3" that is a partition not a complete block device/drive

Quote from NeXTguy

And if so, any idea why sdb must be treated differently?

OK I'm lost on that, you can't treat it differently OMV will use a complete drive to create an array, sdb clearly shows a partition with no raid signature. You can't put a square peg in a round hole

Output of mdadm --detail /dev/md127

NeXTguy · Jul 24th 2020

Okay, thank you. What you write makes sense. The irritating part is that sdb actually is the third disk from the array. It is the same physical disk that was part of the Raid 5 as /dev/sdb before I did the upgrade. Now, why it now reports something different after the upgrade...

cabrio_leo · Jul 24th 2020

NeXTguy Were the RAID drives connected or not connected during the upgrade?
@All Could it be that the upgrade did something (strange) with them? I've read similar error reports here in the board several times after an upgrade.

NeXTguy · Jul 24th 2020

Yeah, I guess that is the rookie mistake I made; I left the array attached when I did the upgrade.

With your help I have a path forward now: I'll simply reformat sdb and add it as a fresh disk to the now degraded array. This will of course require a rebuild but I can live with that.

By the way, cabrio_leo, the NAS in question is in a Node 304 case.

geaves · Jul 24th 2020

Quote from NeXTguy

I'll simply reformat sdb and add it as a fresh disk to the now degraded array

Wipe the drive first, technically you shouldn't have to format it, wipe the drive, raid management, select the raid and hit recover on the menu the drive should be displayed, select it and click ok and the raid should rebuild

ryecoaaron · Jul 25th 2020

The upgrade isn't causing this and you should be able to leave the array connected. I think there is a difference in the mdadm package between Debian 9 and 10 that causes this.

What is the output of: mdadm --detail --scan

If the output of that command doesn't match the array line(s) in /etc/mdadm/mdadm.conf then I think the array won't mount. If they are different, execute:

omv-salt deploy run mdadm

Then reboot and see if it assembles.

Raid missing after upgrade from OMV 4 to 5

mdstat

blkid

fdisk

mdadm

Participate now!

Tags