RAID problem - sdo is assigned to wrong raid array

  • I have a bit of a problem. I had to replace my motherboard and after I replaced it, one of my arrays is messed up. Well actually, teo of them are messed up but one is resyncing.


    i have a three drive array with external usb 3.0 drives - md127. Hoever one of the drives, (currently sdo) is assigned to array md123 and I can't do anythign to fix it.
    when I look at the RAID in the web gui I can't even see md 123 but I see it when I do a cat /proc/mdstat.


    cat /proc/mdstat
    Personalities : [raid6] [raid5] [raid4]


    md123 : inactive sdo[1](S)
    2930265556 blocks super 1.2


    md124 : active raid5 sdj[0] sdl[4] sdk[5] sdh[2] sdi[1]
    7813531648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]


    md125 : active raid5 sde[0] sdg[3] sdf[4]
    2930014208 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]


    md126 : active raid5 sdc[0] sda[3] sdd[2] sdb[1]
    8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
    [====>................] resync = 23.1% (676974720/2930265088) finish=295.8min speed=126918K/sec


    md127 : active raid5 sdp[4] sdm[3]
    5860530176 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U]


    unused devices: <none>


    I need to somehow remove sdo from md123 and then delete md123 and then readd sdo to the md 127 and rebuild it. I think I can do a mdadm --add /dev/md127 /dev/sdo to add it back in but it wont let me do a mdadm --fail /dev/md123 /dev/sdo or a
    mdadm --remove /dev/md123 /dev/sdo so i can't remove it so it won't let me add it.


    Any thoughts and suggestions are welcome.

    • Offizieller Beitrag

    umount /dev/md123
    umount /dev/md127
    mdadm --stop /dev/md123
    mdadm --stop /dev/md127
    mdadm --assemble /dev/md127 /dev/sd[mop] --verbose --force

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • After it rebuilt, now in the front end it is listed as clean, failed and when I do a mdstat it looks not proper.


    cat /proc/mdstat
    Personalities : [raid6] [raid5] [raid4]
    md127 : active raid5 sdo[5](S) sdp[4](F) sdm[3]
    5860530176 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/1] [__U]


    md124 : active raid5 sdj[0] sdl[4] sdk[5] sdh[2] sdi[1]
    7813531648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]


    md125 : active raid5 sde[0] sdg[3] sdf[4]
    2930014208 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]


    md126 : active raid5 sdc[0] sda[3] sdd[2] sdb[1]
    8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]


    unused devices: <none>



    Front End info:


    Number Major Minor RaidDevice State
    0 0 0 0 removed
    1 0 0 1 removed
    3 8 192 2 active sync /dev/sdm
    4 8 240 - faulty spare /dev/sdp
    5 8 224 - spare /dev/sdo

    • Offizieller Beitrag

    Hope you have a backup... Have you rebooted?


    Not sure I would trust /dev/sdp but I would try the following:


    umount /dev/md127
    mdadm --stop /dev/md127
    mdadm --zero-superblock /dev/sdp
    dd if=/dev/zero of=/dev/sdp bs=512 count=1000
    mdadm --assemble /dev/md127 /dev/sd[mop] --verbose --force


    If it does reassemble, your data may be gone.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • No backup, dam dam dam.


    After reboot drive letters changed. Now, the drive letters are M, O and P.



    cat /proc/mdstat
    Personalities : [raid6] [raid5] [raid4]
    md124 : active raid5 sdj[0] sdl[4] sdk[5] sdh[2] sdi[1]
    7813531648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]


    md125 : active raid5 sde[0] sdg[3] sdf[4]
    2930014208 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]


    md126 : active raid5 sdc[0] sda[3] sdd[2] sdb[1]
    8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]


    md127 : inactive sdm[3] sdo[5](S)
    5860531112 blocks super 1.2


    unused devices: <none>




    When I try to do the zero superblock, I get a strange error:


    mdadm --zero-superblock /dev/sdp
    mdadm: Unrecognised md component device - /dev/sdp

    • Offizieller Beitrag

    I would say /dev/sdP is failing. Just to maybe get your data, try:


    umount /dev/md127
    mdadm --stop /dev/md127
    mdadm --assemble /dev/md127 /dev/sdm /dev/sdo missing --verbose --force

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I'm all messed up in the superblock


    mdadm --assemble /dev/md127 /dev/sdm /dev/sdo missing --verbose --force
    mdadm: looking for devices for /dev/md127
    mdadm: cannot open device missing: No such file or directory
    mdadm: missing has no superblock - assembly aborted

    • Offizieller Beitrag

    Try it with the missing.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • mdadm --assemble /dev/md127 /dev/sdm /dev/sdo /dev/sdp --verbose --force
    mdadm: looking for devices for /dev/md127
    mdadm: no RAID superblock on /dev/sdp
    mdadm: /dev/sdp has no superblock - assembly aborted


    Is there a way to copy the superblock from M or O to P ?



    mdadm --examine --scan -v
    ARRAY /dev/md/raid01 level=raid5 metadata=1.2 num-devices=3 UUID=0f5cd663:ce55dc39:214f9bf8:2f4fb26e name=zed:raid01
    spares=1 devices=/dev/sdm,/dev/sdo
    ARRAY /dev/md/raid02 level=raid5 metadata=1.2 num-devices=4 UUID=8ee7fc1c:9dc7207c:cd21a415:b4d6333d name=zed:raid02
    devices=/dev/sdb,/dev/sdc,/dev/sdd,/dev/sda
    ARRAY /dev/md/raid04 level=raid5 metadata=1.2 num-devices=3 UUID=fe65fb09:844373b9:2745ddc9:4bad7f79 name=zed:raid04
    devices=/dev/sdf,/dev/sde,/dev/sdg
    ARRAY /dev/md/raid03 level=raid5 metadata=1.2 num-devices=5 UUID=a8252b01:b3df2b11:b985b9f6:951a2c64 name=zed:raid03
    devices=/dev/sdk,/dev/sdi,/dev/sdh,/dev/sdj,/dev/sdl

    • Offizieller Beitrag

    I meant try it without the missing word. I would pull /dev/sdp.


    mdadm --assemble /dev/md127 /dev/sdm /dev/sdo --verbose --force

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Doh!


    mdadm --assemble /dev/md127 /dev/sdm /dev/sdo --verbose --force
    mdadm: looking for devices for /dev/md127
    mdadm: /dev/sdm is identified as a member of /dev/md127, slot 2.
    mdadm: /dev/sdo is identified as a member of /dev/md127, slot -1.
    mdadm: no uptodate device for slot 0 of /dev/md127
    mdadm: no uptodate device for slot 1 of /dev/md127
    mdadm: added /dev/sdo to /dev/md127 as -1
    mdadm: added /dev/sdm to /dev/md127 as 2
    mdadm: /dev/md127 assembled from 1 drive and 1 spare - not enough to start the array.




    I think the drives are listed as spares for some reason.


    cat /proc/mdstat
    Personalities : [raid6] [raid5] [raid4]
    md127 : inactive sdm[3](S) sdo[5](S)
    5860531112 blocks super 1.2


    md124 : active raid5 sdj[0] sdl[4] sdk[5] sdh[2] sdi[1]
    7813531648 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]


    md125 : active raid5 sde[0] sdg[3] sdf[4]
    2930014208 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]


    md126 : active raid5 sdc[0] sda[3] sdd[2] sdb[1]
    8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]


    unused devices: <none>



    P.S.
    I do appreciate all the assistance. If I could get it up and running for a couple of hours I could move the data off it and that is the hope.

    • Offizieller Beitrag

    Try:
    mdadm --assemble /dev/md127 /dev/sdm /dev/sdo --verbose --force --assume-clean


    If that doesn't work, my last option is much more dangerous. I would only use this as a last resort since it is creating a new array. I have seen this work but also fail.


    mdadm --stop /dev/md127
    mdadm --create /dev/md127 --assume-clean --metadata=1.2 --level=5 --raid-devices=3 /dev/sdm /dev/sdo missing

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I think it's rebuilding but I can't find the filesystem



    cat /proc/mdstat
    Personalities : [raid6] [raid5] [raid4]
    md127 : active raid5 sdr[3] sdq[0] sdo[1]
    5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
    [>....................] recovery = 0.1% (2983500/2930135040) finish=1586.1min speed=30757K/sec

    • Offizieller Beitrag

    It is rebuilding but I would wait until it is done. Then post the output of blkid

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • It's still rebuilding. I have hope. If there was no data the resync would be fast right? As it's taking a zillion hours it must be copying data somewhere. right?



    cat /proc/mdstat
    Personalities : [raid6] [raid5] [raid4]
    md127 : active raid5 sdr[3] sdq[0] sdo[1]
    5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
    [============>........] recovery = 63.4% (1860171252/2930135040) finish=5842.3min speed=3052K/sec

    • Offizieller Beitrag

    Rebuild can be slow with or without data.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • After is rebuilt in my RAID I had three RAIDs for the one RAID and therer are now two filesystems that I can't get rid of and no way to access the data. Thinks for the help, I'm going to try and reformat them outside of OMV and then readd them and reset them up and see how that goes.
    Thanks again for all the help.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!