RAID 1 (Mirror) with LUKS encryption missing after reboot / edit of config.xml

    • OMV 4.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • RAID 1 (Mirror) with LUKS encryption missing after reboot / edit of config.xml

      What happened:
      I recently wanted to install OpenVPN to my OMV v4.1.23-1 and consulted a YouTube video of TechnoDadLife (Link follows). Anyways, after the first steps I had to reboot the system only to find that my Mirror RAID1 (it was encrypted using LUKS) was gone. It's also gone in the encryption section.
      I have read through a couple of threads here, but was yet unable to mount even a single of the two disks, let alone the RAID. After having read these threads, I assume my manual edit of the config.xml and following omv-mkconf fstab was the reason for this error, though I'm clueless why. The exact procedure can be found at this timestamp of the video: https://youtu.be/nMthOobE-8g?t=172 - nothing really complicated especially up to this point.

      When I try to create a new file system, OMV says it'll format the drive(s), which of course I do not want to happen.
      I have already tried to reactivate the RAID by using mdadm -assemble, here's the output:

      Source Code

      1. root@Serverlein:~# mdadm --assemble /dev/md0 /dev/sd[de]
      2. mdadm: no recogniseable superblock on /dev/sdd
      3. mdadm: /dev/sdd has no superblock - assembly aborted
      Here's the output to the Degraded or missing raid array questions

      Source Code

      1. root@Serverlein:~# cat /proc/mdstat
      2. Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
      3. unused devices: <none>
      4. root@Serverlein:~# blkid
      5. /dev/sda1: UUID="2101453d-5973-470f-b374-ca4b8ca56cf3" TYPE="ext4" PARTUUID="2ce08785-01"
      6. /dev/sda5: UUID="262280ab-2e12-4a5f-9b81-86c45670afa6" TYPE="swap" PARTUUID="2ce08785-05"
      7. /dev/sdb: UUID="7012eb61-0ef6-4c55-912a-1a8f524d26e5" TYPE="crypto_LUKS"
      8. /dev/sdc: UUID="f9985f7e-c937-4873-bff9-4fcf41bef625" TYPE="crypto_LUKS"
      9. /dev/mapper/sdb-crypt: LABEL="8TB1" UUID="3ce321fd-a407-4ddf-9fd7-0e7dc50bb461" TYPE="ext4"
      10. /dev/mapper/sdc-crypt: LABEL="8TB2" UUID="55493042-ba3b-4ecf-8430-68bd435d849b" TYPE="ext4"
      11. root@Serverlein:~# fdisk -l | grep "Disk "
      12. Disk /dev/sda: 111.8 GiB, 120034123776 bytes, 234441648 sectors
      13. Disk identifier: 0x2ce08785
      14. Disk /dev/sdb: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
      15. Disk /dev/sdd: 3.7 TiB, 4000753476096 bytes, 7813971633 sectors
      16. Disk /dev/sde: 3.7 TiB, 4000753476096 bytes, 7813971633 sectors
      17. Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
      18. Disk /dev/mapper/sdb-crypt: 7.3 TiB, 8001561124864 bytes, 15628049072 sectors
      19. Disk /dev/mapper/sdc-crypt: 7.3 TiB, 8001561124864 bytes, 15628049072 sectors
      20. root@Serverlein:~# cat /etc/mdadm/mdadm.conf
      21. # mdadm.conf
      22. #
      23. # Please refer to mdadm.conf(5) for information about this file.
      24. #
      25. # by default, scan all partitions (/proc/partitions) for MD superblocks.
      26. # alternatively, specify devices to scan, using wildcards if desired.
      27. # Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
      28. # To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
      29. # used if no RAID devices are configured.
      30. DEVICE partitions
      31. # auto-create devices with Debian standard permissions
      32. CREATE owner=root group=disk mode=0660 auto=yes
      33. # automatically tag new arrays as belonging to the local system
      34. HOMEHOST <system>
      35. # definitions of existing MD arrays
      36. ARRAY /dev/md0 metadata=1.2 name=Serverlein:RAID4TB UUID=fff47ee3:74f5432a:863f8e22:11789248
      37. # instruct the monitoring daemon where to send mail alerts
      38. MAILADDR koenich@koenich.com
      39. MAILFROM rootroot@Serverlein:~# mdadm --detail --scan --verbose
      Display All

      So the two drives for the RAID are /dev/sdd and /dev/sde, both with 4TB capacity (see screenshots below as well). And as said, encrypted using the LUKS / the encryption plugin. Worked perfectly well until this reboot...
      Somehow I have the feeling that OMV "forgot" that these drives are encrypted and needs to be told and afterwards would be fine to recreate the RAID, but I do not know how.

      Here's a screenshot of file systems with the missing RAID:

      And here's one showing the 5 disks of my server. A 128GB SSD as system drive sda, two 8TB drives sdb & sdc (both LUKS encrypted as well) and the two 4TB drives sdd&sde which used to be the LUKS encrypted RAID:
    • First off I have no knowledge of LUKS, but the output from mdstat, blkid and mdadm examine does not display any information about /dev/sd[de] the only real output is from your mdadm assemble that returns no superblock on /dev/sdd

      I remember reading something on here regarding LUKS and Raid, I think you have to turn off LUKS for the Raid before you can do anything.
      Raid is not a backup! Would you go skydiving without a parachute?
    • geaves wrote:

      First off I have no knowledge of LUKS, but the output from mdstat, blkid and mdadm examine does not display any information about /dev/sd[de] the only real output is from your mdadm assemble that returns no superblock on /dev/sdd

      I remember reading something on here regarding LUKS and Raid, I think you have to turn off LUKS for the Raid before you can do anything.
      Thanks for your answer. I think it's pretty weird that no other information about the missing RAID shows up as well.
      But how would I turn off LUKS for the RAID? I would still like to access my data which hides encrypted on those two drives...
    • KOENICH wrote:

      But how would I turn off LUKS for the RAID?
      I don't know, I've never used it, there is another option, in OMV-Extras -> Kernel scroll down and you will see Install SystemRescueCD then an option to reboot to use it, this might help recover the raid as it's all command line and may well bypass LUKS, but again I don't know.

      The superblock error is repairable but only if it's on one drive + using the SystemRescueCD mdadm -D /dev/md0 hopefully will return something, and I suppose you don't have a backup of that raid :)
      Raid is not a backup! Would you go skydiving without a parachute?
    • I am not sure about the best practices here, but using a raid in my opinion the encryption should sit on top of the assembled md0 device, not the underlying members.

      I use luks but just as single drives encrypted merged into a pool.


      This should be right IMO
      RAID->LUKS->FILESYSTEM

      we can wait for other users comments also they use a similar setup
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • RAID 1 (Mirror) with LUKS encryption missing after reboot / edit of config.xml

      subzero79 wrote:

      I am not sure about the best practices here, but using a raid in my opinion the encryption should sit on top of the assembled md0 device, not the underlying members.

      I use luks but just as single drives encrypted merged into a pool.


      This should be right IMO
      RAID->LUKS->FILESYSTEM

      we can wait for other users comments also they use a similar setup
      Honestly I believe that's the way it was configured - I just can't check at the moment. There was only one way to configure Raid and LUKS via OMV and I think that's exactly how it worked.
      I'm wondering at this moment of I could somehow connect only one of the two drives, decrypt it and save the data - usually that should be possible with using Raid 1, but I'm not sure if I can do that with OMV.
    • subzero79 wrote:

      This should be right IMO
      RAID->LUKS->FILESYSTEM

      KOENICH wrote:

      Honestly I believe that's the way it was configured - I just can't check at the moment.
      I did some research on this last night @subzero79 is correct that is the way it should be done, that way you are encrypting the raid and not the two disks.

      One way to check is to look at the output from cat /etc/fstab
      Raid is not a backup! Would you go skydiving without a parachute?
    • RAID 1 (Mirror) with LUKS encryption missing after reboot / edit of config.xml

      subzero79 wrote:

      Is strange from what I can see sdd is missing the raid signature. Reboot the server and put in hastebin the whole dmesg output after boot.

      Should be able to start the raid without one of the disks if is raid1
      Thanks for all your replies. I'm on parental leave at the moment and don't find time to care for my server every day, so please excuse my late answer.

      Where do I find the boot dmesg output as requested?
    • That was easy enough, thanks :P

      I just rebooted the server, here's the dmesg output:

      ---longer than 10.000 chars, so I had to put it in an attachment---


      I didn't do anything after reboot, thus the 8TB drives are not decrypted yet at this moment.

      subzero79 wrote:

      Should be able to start the raid without one of the disks if is raid1
      That's what I thought. But how would I do that? Since at this time I can't decrypt even a single drive in OMV (at least not the web frontend)
      I'm wondering if that would be less work than trying to repair the RAID. I'll probably rsync the two drives in the future, since I'm currently planning to rsync that data to an external OMV as well anyways.
      Files

      The post was edited 1 time, last by KOENICH ().

    • I attached the lsmod output as a txt-file. I also attached the out put for fstab - when reading the whole thread again I realized I was asked for it earlier.

      It was a fresh iso-install.

      Thanks again for caring!
      Files
      • lsmod.txt

        (4.28 kB, downloaded 8 times, last: )
      • fstab.txt

        (1.19 kB, downloaded 6 times, last: )

      The post was edited 1 time, last by KOENICH ().

    • Doesn't show anything relevant i can figure out what's going on. there is not even luks or raid signature for sdd or sde.

      you can try running blkid directly into the device

      blkid /dev/sde or
      blkid /dev/sdd

      see if it returns something
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • geaves wrote:

      The fsatb is incorrect yours is missing this;

      # <<< [openmediavault]

      which should be the last line, OMV adds the fstab entries in # >>> [openmediavault] at the start with # <<< [openmediavault] at the end.
      My bad. I forgot to copy the last line. Sorry.
      # <<< [openmediavault] is actually at the end. I corrected the txt-file in the post above - it really was only the last line missing.

      subzero79 wrote:

      Doesn't show anything relevant i can figure out what's going on. there is not even luks or raid signature for sdd or sde.

      you can try running blkid directly into the device

      blkid /dev/sde or
      blkid /dev/sdd

      see if it returns something
      Doesn't return anything...
    • IPMI is afaik theoretically possible (it's a HP N54L MicroServer), but that's it.

      Given that the two of you haven't experienced this weird behavior I assume it's actually best to try and see what I can do with a Ubuntu live... I'll try if I can at least decrypt one of the two drives (which would be sufficient). I actually have a backup of the content of the two drives, but it's about two weeks old I some of the data has changed in the meantime, so I'll need to see if I can somehow access it again.
      For the future (as said above somewhere) I guess I'll just rsync the two drives and won't trust a soft RAID :(
    • Users Online 3

      3 Guests