RAID 0 Array disappears after every reboot

SeaBee · Jun 13th 2020

ok, so here goes attempt 2.

I recently wiped my old OMV version 4 server and installed OMV 5 (5.5.2-1), which has been going great, until i discovered a bug i started having an issue in OMV 4 where my RAID 0 array would disappear after every reboot and would have run some commands just to make it come back.

My hardware layout is as follows:

- sda - 4TB Storage Disk

- sdb - 3TB storage Disk

- sdc - 2TB Storage Disk

- sdd - random USB i had in the machine

- sde - 2TB Storage Disk

- sdf Boot Drive

Also here are some command i ran which can hopefully help to speed up a fix

cat /etc/mdadm/mdadm.conf

Code

# This file is auto-generated by openmediavault (https://www.openmediavault.org)
# WARNING: Do not edit this file, your changes will get lost.

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# definitions of existing MD arrays
ARRAY /dev/md0 metadata=1.2 name=cb-server.cb-server:CorePool UUID=87674581:aa4aabfe:9312d183:a8ed4906

Display More

fdisk -l | grep "Disk " | grep sd | sort

Code

Disk /dev/sda: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Disk /dev/sdb: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdd: 7.5 GiB, 8053063680 bytes, 15728640 sectors
Disk /dev/sde: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdf: 111.8 GiB, 120040980480 bytes, 234455040 sectors

cat /proc/mdstat

Code

Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid0 sda[0] sde[3] sdc[2] sdb[1]
      10743784960 blocks super 1.2 512k chunks

unused devices: <none>

blkid

Code

/dev/sda: UUID="87674581-aa4a-abfe-9312-d183a8ed4906" UUID_SUB="02249442-0c9b-53e9-0680-f81e34c7a115" LABEL="cb-server.cb-server:CorePool" TYPE="linux_raid_member"
/dev/sdc: UUID="87674581-aa4a-abfe-9312-d183a8ed4906" UUID_SUB="d81adc61-5679-43cb-44d4-897697ae6797" LABEL="cb-server.cb-server:CorePool" TYPE="linux_raid_member"
/dev/sdb: UUID="87674581-aa4a-abfe-9312-d183a8ed4906" UUID_SUB="72eca10b-a959-b3d9-2cf3-31ba469ea87e" LABEL="cb-server.cb-server:CorePool" TYPE="linux_raid_member"
/dev/sdd1: LABEL="OPENMEDIAVA" UUID="48A6-D0E3" TYPE="vfat" PARTLABEL="Microsoft Basic Data" PARTUUID="cee497a1-4ba6-4a90-ac8e-e3d509963f8b"
/dev/sdf1: UUID="0925-8731" TYPE="vfat" PARTUUID="8df33f5f-045b-40e0-9069-417541e8b206"
/dev/sdf2: UUID="e82406a6-2d1e-40a6-8407-39e6244667a3" TYPE="ext4" PARTUUID="270da3fa-168e-4b5a-8660-ad5ee2087e92"
/dev/sdf3: UUID="c7521f20-4245-4779-897a-a168f27daf98" TYPE="swap" PARTUUID="c33c815f-d140-411f-8be2-ebaadf671173"
/dev/sde: UUID="87674581-aa4a-abfe-9312-d183a8ed4906" UUID_SUB="c21c472f-43f2-546f-a21f-e7f300d1d143" LABEL="cb-server.cb-server:CorePool" TYPE="linux_raid_member"
/dev/md0: LABEL="CorePoolData" UUID="6286e1c0-da5a-481e-98e5-5efc57f0463c" TYPE="ext4"

fdisk -l | grep "Disk "

Code

Disk /dev/sda: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: WDC WD40EFRX-68N
Disk /dev/sdc: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: ST2000DL003-9VT1
Disk /dev/sdb: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
Disk model: WDC WD30EFRX-68N
Disk /dev/sdd: 7.5 GiB, 8053063680 bytes, 15728640 sectors
Disk model: Flash Disk
Disk identifier: D803A773-EB4A-4195-970E-1BE07C84840F
Disk /dev/sdf: 111.8 GiB, 120040980480 bytes, 234455040 sectors
Disk model: WDC WDS120G2G0B-
Disk identifier: E3C22D0D-3B9A-4E09-977F-CA637065C94E
Disk /dev/sde: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: WDC WD2003FZEX-0
Disk /dev/md0: 10 TiB, 11001635799040 bytes, 21487569920 sectors

Display More

mdadm --detail --scan --verbose

Code

ARRAY /dev/md0 level=raid0 num-devices=4 metadata=1.2 name=cb-server.cb-server:CorePool UUID=87674581:aa4aabfe:9312d183:a8ed4906
   devices=/dev/sda,/dev/sdb,/dev/sdc,/dev/sde

Any help would be greatly appreciated

*Also yes i know RAID 0 is stupid with the 1 disk dying casing an entire loss of data problem but i have weekly backups which works fine for me*

SeaBee · Jun 13th 2020

also the way i am able to make the array appear after every reboot is by having to issue these 3 commands:

echo 2 > /sys/module/raid0/parameters/default_layout

then...

mdadm --stop /dev/md0

then...

mdadm --assemble --force --verbose /dev/md0 /dev/sd[abce]

and that brings the array back

geaves · Jun 13th 2020

cat /etc/fstab

SeaBee · Jun 13th 2020

here's the output from cat /etc/fstab

Code

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sdf2 during installation
UUID=e82406a6-2d1e-40a6-8407-39e6244667a3 /               ext4    errors=remount-ro 0       1
# /boot/efi was on /dev/sdf1 during installation
UUID=0925-8731  /boot/efi       vfat    umask=0077      0       1
# swap was on /dev/sdf3 during installation
UUID=c7521f20-4245-4779-897a-a168f27daf98 none            swap    sw              0       0
# >>> [openmediavault]
/dev/disk/by-label/CorePoolData         /srv/dev-disk-by-label-CorePoolData     ext4    defaults,nofail,user_xattr,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl      0 2
# <<< [openmediavault]

Display More

geaves · Jun 13th 2020

that looks ok you could try update-initramfs -u

SeaBee · Jun 13th 2020

that gives some firmware missing warming:

Code

update-initramfs: Generating /boot/initrd.img-5.5.0-0.bpo.2-amd64
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8125a-3.fw for module r8169
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168fp-3.fw for module r8169

Also it looks like it could be some network firmware issue and just for some extra info, i don't have any wireless card installed, my server is just connected through the motherboards ethernet port.

geaves · Jun 13th 2020

Wan't expecting an output, the norm is to run that and reboot as what you entering to get the raid back is a module change. But using your first line from post 2 I found this this is beyond me as this seems to relate to Raid 0 and the number of drives within the array so the output from mdadm --detail /dev/md0 would give you more information

SeaBee · Jun 13th 2020

ok, so mdadm --detail /dev/md0 gives:

Code

/dev/md0:
           Version : 1.2
     Creation Time : Fri Jun 12 16:47:37 2020
        Raid Level : raid0
        Array Size : 10743784960 (10246.07 GiB 11001.64 GB)
      Raid Devices : 4
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Fri Jun 12 16:47:37 2020
             State : clean
    Active Devices : 4
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 0

        Chunk Size : 512K

Consistency Policy : none

              Name : cb-server.cb-server:CorePool  (local to host cb-server.cb-server)
              UUID : 87674581:aa4aabfe:9312d183:a8ed4906
            Events : 0

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       8       32        2      active sync   /dev/sdc
       3       8       64        3      active sync   /dev/sde

Display More

geaves · Jun 13th 2020

According to that output and that article the default layout should be 2, that should be set in /etc/default/grub again the article says there should be a line in grub that states GRUB_CMDLINE_LINUX_DEFAULT="raid0.default_layout=2"

SeaBee · Jun 13th 2020

ok so i found another thread that had a similar issue for one of the missing firmware issues (here) and found that:

cd /lib/firmware/rtl_nic

and

wget https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/rtl_nic/rtl8125a-3.fw

then i updated it again with

update-initramfs -u

but im still left with this one missing firmware issue:

Code

update-initramfs: Generating /boot/initrd.img-5.5.0-0.bpo.2-amd64
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168fp-3.fw for module r8169

SeaBee · Jun 13th 2020

Quote from geaves

According to that output and that article the default layout should be 2, that should be set in /etc/default/grub again the article says there should be a line in grub that states GRUB_CMDLINE_LINUX_DEFAULT="raid0.default_layout=2"

is it ok to sudo nano into the grub file, edit it and reboot?

my current grub file looks like this:

Code

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX=""

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"
GRUB_DISABLE_SUBMENU=y

Display More

would it be safe to just replace the GRUB_CMDLINE_LINUX_DEFAULT="quiet" line with GRUB_CMDLINE_LINUX_DEFAULT="raid0.default_layout=2"

geaves · Jun 13th 2020

Quote from SeaBee

would it be safe to just replace the

No, I don't think it's that simple, I'm going to try a VM with Raid 0 and check the settings, but I have not seen this before.

SeaBee · Jun 13th 2020

huh, ok so i took a chance and replaced the GRUB_CMDLINE_LINUX_DEFAULT="quiet" line with GRUB_CMDLINE_LINUX_DEFAULT="raid0.default_layout=2"

then ran the update-grub command and rebooted the machine and it looks like it worked!

the raid array is still present!

also after running the update-initramfs -u command from before only one issue is present, should i make a new post somewhere else or keep trying in this thread?

Code

update-initramfs: Generating /boot/initrd.img-5.5.0-0.bpo.2-amd64
W: Possible missing firmware /lib/firmware/rtl_nic/rtl8168fp-3.fw for module r8169

geaves · Jun 13th 2020

Quote from SeaBee

huh, ok so i took a chance

Why I've just run this up on a VM Raid 0 with 4 drives, the config files are exactly the same as yours and it rebooted without issue, the missing firmware means for whatever reason that module is missing and would probably not affect your system anyway, but I've run that on the VM and it's fine no missing modules, and my kernel is the same.

SeaBee · Jun 13th 2020

Quote from geaves

Why I've just run this up on a VM Raid 0 with 4 drives, the config files are exactly the same as yours and it rebooted without issue, the missing firmware means for whatever reason that module is missing and would probably not affect your system anyway, but I've run that on the VM and it's fine no missing modules, and my kernel is the same.

weird, might just be something up with my hardware or something.

well at least the main issue is fixed.

Thanks a ton for your help

RAID 0 Array disappears after every reboot

SeaBee Jun 13th 2020

SeaBee Jun 13th 2020

Participate now!

Tags