Suddenly trouble with root filesystem getting destroyed

villeneuve · 12. Januar 2018

Hello,
I recently cloned my OMV-installation onto another USB-stick. To do that I bootet into a live-cd and used the command if=/dev/sdx of=/dev/sdy bs=4K . It all went fine and worked for weeks.

Yesterday though the system would not boot anymore. It gave me the following error:
error: file '/boot/grub/i386-pc/normal.mod' not found.
Entering rescue mode...

When I tried to fix the problem using a Linux live-cd I discovered that the file system on the USB-stick was damaged and I could not repair it with fsck either.
I then went for the easiest solution and cloned the old USB-stick again onto the new one. OMV then bootet again.

I then decided to update it but could not do it since it told me that "/var/cache/apt/archives/partial" was read only. I already assumed that I'm back into trouble and wondered if the root-filesystem is damaged again. To find out I initiated a reboot and sure enough I received the "error: file '/boot/grub/i386-pc/normal.mod' not found."-error again.

Before all this I had tested the new USB-stick with H2testw for errors, but it looked all good.

Does anyone have any clue what's going on?

Thanks for reading!

*EDIT* I once again checked the USB-stick with H2testw and it tested it successfully.

gderf · 12. Januar 2018

Are you using the Flash Memory plugin on OMV? Your USB stick will likely be worn out without the plugin being used. But you seem to be having problems much sooner that I would expect, if wear was the problem.

Are your USB sticks well known reliable brand names?

villeneuve · 12. Januar 2018

Yes, I was using Flash Memory Plugin 3.5.
As I wrote I tested the new USB-stick before all that and again today and it is certainly not worn out, otherwise it H2testw would show that.
The new USB stick is from Supertalent.

villeneuve · 17. Januar 2018

After trying a lot of things I did a clean install to the new USB drive. After the first boot, which went fine, I was busy with other stuff, so the system just sat idle. When I came back tty1 showed the following:

EXT4-fs error (device sdc1): ext4_mb_generate_buddy:758: group 3, block bitmap and bg descriptor inconsistent: 18977 vs 184 free clusters
Aborting journal on device sdc1-8.
EXT4-fs (sdc1): Remounting filesystem read-only
EXT4-fs error (device sdc1): ext4_journal_check_start:56: Detected aborted journal

I'm out of ideas now and will try another USB-stick tomorrow.

tkaiser · 17. Januar 2018

If you cloned an old installation with dd you copy all filesystem errors that were present at the source (if there were any). If you're using USB front ports that suffer from cable/contact problems (cables too long affecting signal quality, shielding issues, SuperSpeed contact issues in case you're using USB3 products) you get data corruption on the wire. To check for this potential issue you could run an integrity check (use the search bar in the upper right and use 'dpkg verify')

My personal recipe for USB pendrives is using only good brands (those vendors that produce their own NAND dies and also controllers and manufacture this to retail products --> Samsung, SanDisk, Toshiba, Transcend), choose USB3 products since 'fast enough' but operate it on USB2 ports to avoid USB3 SuperSpeed contact hassles (the USB3-A receptacle design is IMO a fail since the chance to run into contact hassles on SuperSpeed data lines is magnitudes higher than with the older data modes, this problem is only addressed with USB-C now)

Jetzt mitmachen!