Lost RAID upgrading from v0.5x to v1.0

  • I noticed that my OpenMediaVault (OMV) version was fairly out of date, so I decided to upgrade. I performed an in-place upgrade to v1.0 but, at some point in the upgrade, I lost network access. After doing some reading, I found that if I manually create the "/run/network" directory, I would get my network back. This worked, then completed the upgrade (can't remember the command now), but I lost the RAID. For whatever reason, it wouldn't assemble my RAID any more. Eventually, I decided to save off some config files, unplug my RAID, and upgrade with the v1.9 ISO. It loaded just fine. When I shut it down, plugged in my data drives, and tried to assemble the RAID, it was no better. I was finally able to assemble it with 3 of the 4 drives into the RAID 5, but I couldn't mount it. I've exhausted everything I've found to try from the web or my own experience, and am half sick with stress that I've lost my data. Below is some information that will hopefully prove useful. I think there may also be issues with the superblock of either the RAID or individual drives. Before trying OMV v1.9, I tried to restore a superblock for my 4th hard drive, that did not have an issue prior to the recent chaos, but it didn't work. I honestly don't recall what the error was at this point. Sorry for the wall of text, but is anyone able to help?


    mdadm --examine /dev/md127


    mdadm --detail /dev/md127


    mdadm.conf

    Quote


    ARRAY /dev/md127 metadata=1.2 name=OMV-NAS:127 UUID=353afb78:aae4adf0:88709556:4f6c72ff


    blkid


    fdisk -l


    /etc/fstab (just the relevant line for the array)

    Quote


    UUID=f5c065f5-4e18-8600-bd2c-9d4faabc392e /media/raid-storage ext4 defaults 0 2


    mount -a

    Quote


    mount: wrong fs type, bad option, bad superblock on /dev/md127,
    missing codepage or helper program, or other error
    In some cases useful info is found in syslog - try
    dmesg | tail or so

  • I tried:
    mdadm --create /dev/md127 -v -l 5 -n 4 /dev/sdb /dev/sdc /dev/sdd /dev/sde


    That would fail, as it seems something is wrong with /dev/sde, so I tried:
    mdadm --create /dev/md127 -v -l 5 -n 4 /dev/sdb /dev/sdc /dev/sdd missing


    That created it fine with 3 out of 4 drives, but I couldn't do anything else with it.

  • Using the --create flag is a bad idea. You created a new array overwriting previous information. It won't even know there is a filesystem on it as you have found. Photorec may be your only choice now. Just a note, you should only use the --assemble flag when fixing an array. If it doesn't work, using --force can help.

    omv 5.5.12 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.2
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • That wasn't the first attempt I made to recreate the array, including using the GUI in OMV that wouldn't even list the RAID, but it was the latest and that's all that matters. Thanks for the advice, and I'll try to see if Photorec can recover it. If nothing else, it helps quite a bit to know that any further attempts at assembling it will fail, and I can stop wasting my time.


    As a side note, I accept full responsibility for my relative ignorance in trying to get my RAID operational again, but the reason it was even required is very upsetting, starting with a common bug in the upgrade process.

  • As a side note, I accept full responsibility for my relative ignorance in trying to get my RAID operational again, but the reason it was even required is very upsetting, starting with a common bug in the upgrade process.


    What was the bug?

    omv 5.5.12 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.2
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • During the upgrade, it lost networking. There are several posts on the web but, essentially, the "/run/network" directory didn't exist anymore. Once I found that, I manually created the directory, ifup, and all was good. If you reboot, the directory disappears again and has to be recreated. There appeared to be two reported ways to get the directory to stay:

    1. Create the directory as root (not a regular account using sudo)
    2. Use omv-firstaid to setup networking again.


    I had done both before rebooting, so I can't say which method worked, but it did. I was able to complete the upgrade with the "-f" option after that. What started me down the rabbit hole was thinking that it's a good idea to upgrade a working system to the next version. During the upgrade, it killed my networking, halting the upgrade. After getting it back, my RAID played hide-and-seek with me and I lost. In fact, I lost 2TB of data. I tried getting it back through the GUI, but it would lock up every time I tried to look at the storage. Everything I read about people that have lost their RAID, had some success with getting it back using the "create" option, although there were many others I tried first. I hadn't read where that killed the RAID, but you live and learn. It may be some freakish mix of hardware and software issues, but that it was working prior to all this makes me wish I hadn't upgraded at all. I'm trying to get it back through Photorec and, though the software is great, the contents will still never be the same even if I get them back. I could spend months renaming files alone, presuming I ever knew the original name. I'm still desperately clinging to any hope of getting the RAID back in shape, but that hope is swiftly fading.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!