Help Mounting Failing Drive One Last Time - Filesystem Corrupt?

  • I had a drive that I forgot to monitor and the sucker died on me without warning. I've been running fsck commands on it but I've made it no better or possibly worse. My system doesn't have any RAID since it is just media files which I can get back (it will just be a pain to do that).


    What I'm hoping is that you fine folks can help this linux novice out on a command/method I can run to at least get this drive to mount one last time to get data from it.


    Right now, OMV warns me that manual fsck needs run every time I reboot. I run it on the device (by the way, since the /dev/sdX1 changes every time I reboot, is there another way to identify which drive is the corrupt one at the command line by seeing the drive label?)


    Errors I have seen:


    I want this drive out of my system and in the garbage as fast as possible, however, I'd really like a shot at getting any remaining data that is on there (whatever I did not already corrupt further by running these fsck -fy -c -t commands and saying yes to everything)...


    I did run a dumpe2fs | grep superblock command on the file system and there were a ton of backup superblocks it found, which may be good news and somewhere to start?


    Right now, openmediavault does boot and see the drive and the label, but it does not mount the drive, and throws a wicked error when I try and mount it. I don't want to blindly try throwing fsck and other commands at the drive anymore not knowing exactly what I am doing. Any suggestions on what I can do (even by removing the physical drive and booting it into a standalone Debian live CD system or something to try and get at it)...I'll try anything.


    Help?

  • I have some more details so I am giving this a BUMP in the hopes some of you can help.


    I've done a lot of tinkering with this drive, right now I have it yanked out of the OMV box and I'm trying to access it on a Debian Live CD system.


    Debian sees the drive but refuses to mount it. I've run so many fsck commands on this drive. Right now on the latest fsck it seems to be getting an "error reading block" on every block and doing a force rewrite. It's up to block 610XXXXX and still going strong after hours of running. I have no idea what it's doing but suspect maybe the "force rewrite" is just basically deleting all my data.


    The file system downgraded to ext2 at some point from the ext4 it was set to. Debian sees it as an ext2 file system now but still unable to mount it.


    Running dumpe2fs did reveal there were a lot of superblocks still intact.


    Not sure what to do next. I am not trying to save the drive long-term, just one more mount to allow me to copy as much of the data off as I can.

    • Offizieller Beitrag

    Assuming you have the space to do it (normally when the drive first started going bad), I would make an exact image of the drive using dd. Then you work with the image instead of the failing drive.


    Here is how I would image the drive (replace sdX1 with the partition on the drive):
    dd if=/dev/sdX1 of=/path/to/sdX1.dd bs=1M iflag=direct conv=noerror,sync


    Then you can mount that image just like a hard drive:
    mkdir -p /mnt/recover
    mount -o loop /path/to/sdX1.dd /mnt/recover

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thanks for the reply. Since this is a 3 TB drive I don't have an empty 3TB at the moment but I'm working on finding some space or will just buy another drive.


    Given all the damage I may have done already, seeing as the file system somehow downgraded to ext2, and I've been letting fsck run on -y and rewrite all kinds of blocks over and over (for hours), is it possible that I won't even have any data left in there?


    If you think there's still hope, I will definitely run the dd command above and work with the image. Is there any chance that I'll run the DD and the image will be unable to be mounted? I did start the dd command and it was definitely able to read the bad drive, but I aborted it when I realized I would need 3TB free for the img file to build.

    • Offizieller Beitrag

    Being downgraded to ext2 shouldn't hurt it (I think). If the data is important, I would try the dd route. Otherwise, it is going to take a long time. The image should be mountable in this case. Not sure what you chances of recovery are but the dd image won't degrade anymore. You can even fsck the dd image and maybe have better luck.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thanks so much for hanging with me. Does this tell us anything?


    Keep in mind this was an ext4 filesystem, which somehow now has Debian thinking it is an ext2 filesystem (a message popped up that the filesytem type changed somewhere along the way of running fsck's).


    root@debian:/home/user# sudo mount /dev/sdd1 /mnt
    mount: wrong fs type, bad option, bad superblock on /dev/sdd1,
    missing codepage or helper program, or other error
    In some cases useful info is found in syslog - try
    dmesg | tail or so



    root@debian:/home/user# dmesg | tail
    [ 4771.413017] ata5.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 24
    [ 4771.413018] res 51/04:00:00:00:00/00:00:00:00:00/a0 Emask 0x1 (device error)
    [ 4771.413022] ata5.00: status: { DRDY ERR }
    [ 4771.413024] ata5.00: error: { ABRT }
    [ 4771.454173] ata5.00: configured for UDMA/133
    [ 4771.454179] ata5.00: device reported invalid CHS sector 0
    [ 4771.454185] ata5: EH complete
    [ 7216.391972] EXT2-fs (sdd1): error: can't find an ext2 filesystem on dev sdd1.
    [17557.321323] EXT2-fs (sdd1): error: ext2_check_descriptors: Block bitmap for group 128 not in group (block 0)!
    [17557.321328] EXT2-fs (sdd1): group descriptors corrupted

    • Offizieller Beitrag

    An ext3/4 is basically just an ext2 filesystem with journaling. The files are still stored the same way.


    That error message means the drive is in bad shape. Not much we can do to help. The professional services that restore files from failing hard drives would put the platters from your drive in a new drive housing to recover the files. That is an expensive option but sometimes the only option.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Got it. So is there still merit then, assuming I can get the dd to run all the way through without the hardware failing, to trying to image that drive to a file where I may be able to rescue some of the data? What I think has been happening is that the drive has been sort of just hard-erroring out, so my assumption is that during the dd, trying to move 3 TB of data to an image on another drive where I make space...it'll be tough to get through that dd without crashing. Worth a shot though?

    • Offizieller Beitrag

    Yep, I think it is worth a shot. The worse the drive is, the longer it takes. I did this with a 1 TB drive a couple years ago and it took three days to make the dd image but it was in really bad shape.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!