Missing Data on Raid1 RPi

  • Recently my raspberry Pi power was disconnected without warning. When it powered back on, it rebooted in 'emergency mode'. After going through the steps to comment out the extra drive within the /etc/fstab file, I was able to boot up the Pi but my OMV RAID 1 would not mount. When tried to mount I was prompted with an error message. After browsing the Internet I ran fsck on the /dev/md0 mount and it completed successfully. My RAID 1 array was now able to be mounted but I could not see any of my data. The drive space of the array was also correctly reporting so I know there is data on the array. I was hoping you can help me with walking me through how to make my data browsable again. After fixing this, I plan on copying the data to a separate drive not using the RAID setup anymore. I will use Rsync instead.


    Here is my output from the commands:

    pi@raspberrypi:~ $ sudo cat /proc/mdstat

    Personalities : [raid1]

    md0 : active raid1 sdb1[0]

    3906885440 blocks super 1.2 [2/1] [U_]

    bitmap: 0/30 pages [0KB], 65536KB chunk


    unused devices: <none>


    pi@raspberrypi:~ $ sudo blkid

    /dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="4BBD-D3E7" TYPE="vfat" PARTUUID="738a4d67-01"

    /dev/mmcblk0p2: LABEL="rootfs" UUID="45e99191-771b-4e12-a526-0779148892cb" TYPE="ext4" PARTUUID="738a4d67-02"

    /dev/sda1: UUID="4e020b87-9fc1-db8f-613d-8c256e9f486d" UUID_SUB="6dde817d-aeb2-4cba-818d-0e9cfc4841b6" LABEL="raspberrypi:0" TYPE="linux_raid_member" PARTUUID="a7b373ff-8971-4029-b3fd-987efc076ecb"

    /dev/sdb1: UUID="4e020b87-9fc1-db8f-613d-8c256e9f486d" UUID_SUB="15ebe1f2-5611-0d88-3e1e-e014279281ec" LABEL="raspberrypi:0" TYPE="linux_raid_member" PARTUUID="a25a896f-6ba8-be48-8d08-d501dc308228"

    /dev/md0: UUID="f7f6036d-aab8-49e7-8f45-2ac1cc870e02" TYPE="ext4"

    /dev/mmcblk0: PTUUID="738a4d67" PTTYPE="dos"


    pi@raspberrypi:~ $ sudo fdisk -l | grep "Disk "

    The backup GPT table is corrupt, but the primary appears OK, so that will be used.

    Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram1: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram2: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram3: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram4: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram5: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram6: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram7: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram8: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram9: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram10: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram11: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram12: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram13: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram14: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/ram15: 4 MiB, 4194304 bytes, 8192 sectors

    Disk /dev/mmcblk0: 29.8 GiB, 32010928128 bytes, 62521344 sectors

    Disk identifier: 0x738a4d67

    The backup GPT table is corrupt, but the primary appears OK, so that will be used.

    Disk /dev/sdb: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors

    Disk model: nal USB 3.0

    Disk identifier: 95273E32-0319-6240-8C0F-0DCAE21C8092

    Disk /dev/md0: 3.7 TiB, 4000650690560 bytes, 7813770880 sectors

    Disk /dev/sda: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors

    Disk model: nal USB 3.0

    Disk identifier: 9EF89B64-6007-4015-B522-97DCDA041367


    pi@raspberrypi:~ $ sudo cat /etc/mdadm/mdadm.conf

    # This file is auto-generated by openmediavault (https://www.openmediavault.org)

    # WARNING: Do not edit this file, your changes will get lost.


    # mdadm.conf

    #

    # Please refer to mdadm.conf(5) for information about this file.

    #


    # by default, scan all partitions (/proc/partitions) for MD superblocks.

    # alternatively, specify devices to scan, using wildcards if desired.

    # Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.

    # To avoid the auto-assembly of RAID devices a pattern that CAN'T match is

    # used if no RAID devices are configured.

    DEVICE partitions


    # auto-create devices with Debian standard permissions

    CREATE owner=root group=disk mode=0660 auto=yes


    # automatically tag new arrays as belonging to the local system

    HOMEHOST <system>


    # definitions of existing MD arrays

    ARRAY /dev/md0 metadata=1.2 name=raspberrypi:0 UUID=4e020b87:9fc1db8f:613d8c25:6e9f486d


    pi@raspberrypi:~ $ sudo mdadm --detail --scan --verbose

    ARRAY /dev/md0 level=raid1 num-devices=2 metadata=1.2 name=raspberrypi:0 UUID=4e020b87:9fc1db8f:613d8c25:6e9f486d

    devices=/dev/sdb1

  • Glad you are going to stop using raid. This is exactly why we tell people to not use raid with usb and sbcs.


    Your array is assembled with one drive according to the output. This means you should be able to get the data off of it. I would imagine the filesystem just isn't mounted. Try: sudo mount -a

    omv 5.6.9 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • My RAID 1 array was now able to be mounted

    Out of curiosity: how did you set your RAID array?

    2x independent USB Hdds?

    1x USB box that holds the 2x Hdds in Single mode?

  • ryecoaaron


    Thanks for replying.


    Here is the output:

    pi@raspberrypi:~ $ sudo mount -l

    /dev/mmcblk0p2 on / type ext4 (rw,noatime,nodiratime) [rootfs]

    devtmpfs on /dev type devtmpfs (rw,relatime,size=1827468k,nr_inodes=97606,mode=755)

    sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)

    proc on /proc type proc (rw,relatime)

    securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)

    tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)

    devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)

    tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)

    tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)

    tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)

    cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)

    cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)

    none on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)

    cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)

    cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)

    cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)

    cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)

    cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)

    cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)

    cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)

    cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)

    debugfs on /sys/kernel/debug type debugfs (rw,relatime)

    sunrpc on /run/rpc_pipefs type rpc_pipefs (rw,relatime)

    mqueue on /dev/mqueue type mqueue (rw,relatime)

    systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)

    nfsd on /proc/fs/nfsd type nfsd (rw,relatime)

    configfs on /sys/kernel/config type configfs (rw,relatime)

    tmpfs on /tmp type tmpfs (rw,relatime)

    /dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro) [boot]

    /dev/mmcblk0p2 on /var/folder2ram/var/log type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/log type tmpfs (rw,relatime)

    /dev/md0 on /srv/dev-disk-by-id-md-name-raspberrypi-0 type ext4 (rw,relatime,stripe=64,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group)

    /dev/mmcblk0p2 on /var/folder2ram/var/tmp type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/tmp type tmpfs (rw,relatime)

    /dev/mmcblk0p2 on /var/folder2ram/var/lib/openmediavault/rrd type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/lib/openmediavault/rrd type tmpfs (rw,relatime)

    /dev/mmcblk0p2 on /var/folder2ram/var/spool type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/spool type tmpfs (rw,relatime)

    /dev/mmcblk0p2 on /var/folder2ram/var/lib/rrdcached type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/lib/rrdcached type tmpfs (rw,relatime)

    /dev/mmcblk0p2 on /var/folder2ram/var/lib/monit type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/lib/monit type tmpfs (rw,relatime)

    /dev/mmcblk0p2 on /var/folder2ram/var/cache/samba type ext4 (rw,noatime,nodiratime) [rootfs]

    folder2ram on /var/cache/samba type tmpfs (rw,relatime)

    tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=391912k,mode=700,uid=1000,gid=1000)


    I am still unable to browse or see any files or folders on the RAID array. Even though the RAID is mounted.


    When I try to create a new Shared Folder, when I browse to the path with the /dev/md0/ device selected, it does not list any folders.


    Any ideas on how I can restore making these files visible/browsable again? I would then just copy them to another drive and break the RAID and start over.


  • Any ideas on how I can restore making these files visible/browsable again?

    Check to see that they are there first - sudo ls -al /srv/dev-disk-by-id-md-name-raspberrypi-0

    I would then just copy them to another drive and break the RAID and start over.

    Raid is already broken since you only have one of two drives in the array.

    omv 5.6.9 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • geaves - I executed 'sudo mount -a' but it did not provide any output so I ran the mount list command.


    ryecoaaron - I disconnected one of the drive to run TestDisk on it to see if I can recovery any of the data.


    pi@raspberrypi:~ $ sudo ls -al /srv/dev-disk-by-id-md-name-raspberrypi-0

    ls: cannot access '/srv/dev-disk-by-id-md-name-raspberrypi-0/lost+found': Bad message

    total 8

    drwxr-xr-x 3 root root 4096 Apr 27 13:53 .

    drwxr-xr-x 7 root root 4096 Apr 12 2020 ..

    d????????? ? ? ? ? ? lost+found


    When I run TestDisk it states the filesystem is damaged.


    Is there anyway I can repair the filesystem?

  • Your filesystem is not in a good state. That is why you don't see any data. Did you create mdadm create by any chance trying to fix it? If you did, the data is probably gone.

    omv 5.6.9 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • geaves - Thank you, I did not know that.


    ryecoaaron - I do not recall executing that command. Is that part of the fsck process? Could I do something in order to rebuild the filesystem? TestDisk allows you to rebuild the filesystem using the SuperBlock. Is that something I should do on one of the drives?

  • Is that part of the fsck process?

    No.


    Could I do something in order to rebuild the filesystem?

    Sometimes. When raid is underneath, we don't know if an assemble went bad and the disk you are testing against doesn't have all of the info.


    TestDisk allows you to rebuild the filesystem using the SuperBlock. Is that something I should do on one of the drives?

    You can try it. I have never used testdisk to do that. Do whatever you are comfortable with. I would just restore from backup since most raid things take forever. But you probably can't do that.

    omv 5.6.9 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.5
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I powered down my Pi and my external drive bays. I powered back on my external drive bays and then my Pi. I confirmed within OMV the RAID was there, but it was marked as degraded due to me disconnecting one of the drives before. Do you know how I can enable the mirroring of the drives? Or does this happen automatically?


    I ran sudo mount -a and this time I received the following output:

    mount: /srv/dev-disk-by-id-md-name-raspberrypi-0: mount(2) system call failed: Structure needs cleaning.


    Any suggestions on what commands I should run to clean the structure?


    Thank you in advance for all the help, everyone. I am new at this and I am finding I am running into a lot of issues which I am learning quickly to avoid in the future.

  • I ran the following commands again and luckily the lost+found directory was created with all of my data within them.


    pi@raspberrypi:~ $ sudo fsck -C -p /dev/md0

    fsck from util-linux 2.33.1

    /dev/md0 was not cleanly unmounted, check forced.

    /dev/md0: 94140/244187136 files (2.0% non-contiguous), 765759631/976721360 blocks

    pi@raspberrypi:~ $ sudo mount -a


    I am very lucky and I am immediately copying this data off to a different drive.


    Thank you all for the help and support!

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!