Killed EXT4 Filesystem any ideas ?

Okay, so today I totally screwed ~5TB of data .

I switched from onboard sata to an IBM1015-IT, which worked fine at first but when I was booting OMV up I noticed some "EXT4-fs..." errors popping up. fsck aborted after ~17% and wanted me to run it manually. I though "let's fix this real quick, can't be that bad right ?"....boy was I wrong.
After some trial and error getting fsck to run properly I got to the point where it ran out of memory. So I added more RAM and a bigger swapfile and ran fsck again. No luck still not enough memory, at that point I figured I was screwed because I wasn't even able to mount the FS anymore, which worked before even with those errors.

I don't think there's anything I can do at this point right? I guess I'll have to format and start all over again .

Thanks for the tips. I tried a bunch of things before I gave up, gparted/partedmagic/testdisk all failed to get me even into the filesystem. Didn't know photorec, I'll remember that for next time.
I decided to start from scratch, good time to test out kralizec too. I was smart enough to fill up my old 2TB drive, with which I started this project with, about a year ago as a cold backup, has been lying around ever since. So I guess i got that going for me.
I switch to xfs too, I hope it's a bit more stable than ext4. I'm still not sure why the filesystem went bonkers, could be because of power outages, we've had a few of those. Sh*t happens.

Anyway, whats your strategy in terms of filesystem structure ?
I've got a raid5 as base and on top of that two logical volumes, one for homedirs and the other for everything else.

Zitat von davidh2k

Why use LVM at all?

For me this just adds an unneccessary layer of complexity, and doesn't serve any goods in return in a home Environment.

Greetings
David

No particular reason. I just liked it. The ability to split and manage your storage with a few clicks is kinda nice. On top of that you could, for example, combine multiple raid arrays to one big storage pool.

Zitat von ryecoaaron

You can split and manage your storage with shared folders just as easily. If you lost an array for some reason, everything is gone with lvm on top. Not an lvm fan here

I thought everybody loved lvm .

But yes, it has its downsides.

Okay, can someone explain this to me?

I was watching some videos that I copied back to the new filesystem yesterday. They were all skipping and some even restarting for no apparent reason. I tested it from another pc, same thing.
I had a video file that I downloaded locally which I later renamed and copied to the new filesystem. The local copy played fine, the one on the filesystem did not. I compared MD5 and CRC32 they didn't match. So far so good, could've been a bad transfer or something.
Then while I was listening to music from the new filesystem I noticed that it started skipping and sometimes it seemed there were parts of other songs mixed in. I compared checksums again, all mismatched. So I though "okay let's try this again". I Deleted the music from the filesystem and copied it back again, checksums were fine. I cleared my playlist, imported the songs again and started listening. After a few minutes it happened again, skipping/parts of other songs. Compared the checksum of the file I had verified before again, mismatch.
I looked at video files randomly and all had been corrupted to some degree, some restarted after a few seconds others had weird artifacts.

I've deleted the xfs filesystem again and this time created it via cli after a fresh boot. Copying files back right now, music is fine so far.

Problem is still there. Syslog shows a lot of these.

Aug 17 17:49:27 kralizec kernel: [96129.124383]  [<ffffffffa02db731>] ? xfs_corruption_error+0x54/0x6f [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124394]  [<ffffffffa0302940>] ? xfs_da_read_buf+0x1a/0x1f [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124404]  [<ffffffffa030282f>] ? xfs_da_do_buf+0x47e/0x53c [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124413]  [<ffffffffa0302940>] ? xfs_da_read_buf+0x1a/0x1f [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124422]  [<ffffffffa0302940>] ? xfs_da_read_buf+0x1a/0x1f [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124430]  [<ffffffffa0302940>] ? xfs_da_read_buf+0x1a/0x1f [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124434]  [<ffffffff8110bb84>] ? __d_lookup_rcu+0x34/0xfe
Aug 17 17:49:27 kralizec kernel: [96129.124442]  [<ffffffffa0302690>] ? xfs_da_do_buf+0x2df/0x53c [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124451]  [<ffffffffa02f0b69>] ? xfs_attr_leaf_get+0x2d/0x9d [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124459]  [<ffffffffa02f0d4f>] ? xfs_attr_get_int+0xaf/0xd8 [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124468]  [<ffffffffa02f0df2>] ? xfs_attr_get+0x7a/0x98 [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124477]  [<ffffffffa032894d>] ? xfs_get_acl+0xf3/0x146 [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124486]  [<ffffffffa03289ba>] ? xfs_xattr_acl_get+0x1a/0x57 [xfs]
Aug 17 17:49:27 kralizec kernel: [96129.124488]  [<ffffffff81113bfc>] ? getxattr+0x84/0x126
Aug 17 17:49:27 kralizec kernel: [96129.124490]  [<ffffffff8111369e>] ? generic_getxattr+0x55/0x5d
Aug 17 17:49:27 kralizec kernel: [96129.124492]  [<ffffffff81113b6d>] ? vfs_getxattr+0x8c/0x97
Aug 17 17:49:27 kralizec kernel: [96129.124494]  [<ffffffff81113c1b>] ? getxattr+0xa3/0x126
Aug 17 17:49:27 kralizec kernel: [96129.124497]  [<ffffffff810eb7b4>] ? kmem_cache_free+0x2d/0x69
Aug 17 17:49:27 kralizec kernel: [96129.124499]  [<ffffffff81106413>] ? user_path_at_empty+0x53/0x7b
Aug 17 17:49:27 kralizec kernel: [96129.124502]  [<ffffffff810fe5a1>] ? sys_newstat+0x23/0x2b
Aug 17 17:49:27 kralizec kernel: [96129.124504]  [<ffffffff81114347>] ? sys_getxattr+0x42/0x5f
Aug 17 17:49:27 kralizec kernel: [96129.124506]  [<ffffffff81355452>] ? system_call_fastpath+0x16/0x1b
Aug 17 17:49:27 kralizec kernel: [96129.124508] XFS (dm-2): Corruption detected. Unmount and run xfs_repair
Aug 17 17:49:27 kralizec kernel: [96129.124518] ffff8800d7a46000: 00 00 10 00 00 00 02 00 00 00 00 00 00 00 00 00  ................
Aug 17 17:49:27 kralizec kernel: [96129.124520] XFS (dm-2): Internal error xfs_da_do_buf(2) at line 2097 of file /build/linux-eKuxrT/linux-3.2.60/fs/xfs/xfs_da_btree.c.  Caller 0xffffffffa0302940

Rebuilding Raid right now. But I think I might have a damaged drive. The drive in question has a very high "Command timeout" counter. See http://pastebin.com/raw.php?i=JYkWMWed

Data corruption is still happening. I've pulled the drive that I think is broken now, it wasn't part of the raid anymore but maybe it still did something weird. I've found an interesting article that describes somewhat what I've been experiencing. https://peterkieser.com/2013/0…-causing-data-corruption/
I have to do some more testing but right now it looks like it really is a controller problem. Passing through the drives directly would fix this I guess, but then I'd lose SMART data.

Has anyone tried to install Openmediavault alongside Proxmox ? That would be sweet. I've installed Proxmox-testing on debain wheezy before, the only question is if it would peacefully co-exist with OMV.