Posts by bbddpp

For about 3 years now, I have simply been running my OMV server like a computer - Single independent ext4 formatted drives. When one gets full, I add another. Repeat.

Performance has been fine, and I like the simplistic approach like owning a PC. Best of all, no space is wasted. If I have 25 TB of hard drives, I get 25 TB of file storage.

However, lately I have had drives suddenly crashing on me, with data loss, and it's made me reconsider RAID.

If I have 25 TB of data that I want to store, how many drives and how much space would I need to set up RAID? I think the other daunting thing about RAID is that there are so many styles and types (not to mention hardware and software) and I can never seem to find the answer to what the right kind of RAID is for me.

If I switched I would expect that if one of my drives crashed, I could pull it, put in a replacement, and 0 data would be lost. Otherwise, I don't see a huge benefit in doin git.

So, which RAID is right for me under OMV (storing and streaming media files in my home, and downloading new ones via plugins is all my OMV box does)? And how much actual drive space would I need to store 25TB of actual data?

Thanks, I love OMV and appreciate the advice.

Also want to say thanks about flashmemory. Had no idea it existed and just installed it. Not sure that will fix the fact my SSD is bad but at least I know for the future once I get this new SSD running. Thanks so much.

I think I will attempt to clonezilla everything to a new drive tonight unless there is a way to easily copy all my config over to a new OMV installation on another drive. If there is, I'll just do it that way. I'm just trying to avoid having to reconfigure everything.

Thanks. I will have to look into the flashmemory plugin. I probably did not and it sounds like this is something I should have done.

I'll try clonezilla again with the drives plugged into another system and a Live CD. The Clonezilla reboot in my OMV box didn't want to image my OMV drive for some reason. I realized I need to resize the partition before I do that anyway, since the old OS SSD is 300GB and this new one is only 120GB. So I think I first need to use Parted Magic to resize my current OMV drive to 120GB, then Clonezilla to image it and move the image to a a new drive.

I had considered just starting fresh with a new OMV install on the new drive, but then I figure I need to do all this again manually (right?)

- Re-import all existing ext4 drives (each drive is on its own right now, no RAID)
- Redo all shares
- NZBGet Config and Setup
- Sonarr config and setup
- Couchptoato config and setup
- Rebuild my MySQL database for Kodi and start over

I love a fresh install, but I'm worried that there's a lot I could potentially lose or at minimum a LOT of data to rebuild especially for all my file downloading plugins. I'm sure there's more.

I just figured, rather than have to redo all that, easier to move everything to a new healthy drive and keep the files and all the setup and config as-is.

If there's an in-between here or an easy way to create a new OMV installation on a new SSD and then just import all of OMV's config and plugins and all their current data as if I never changed the drive, I'd do that too.

Thanks.

The SSD I have been using to boot OMV is reporting errors in SMART and bad sectors. I have a new SSD plugged into the system that I'd essentially like to copy all of OMV over to, then shutdown, remove the current boot SSD, and have this one just act as the new OMV drive.

I attempted to use Clonezilla natively via the plugin (one-time reboot directly into Clonezilla) but during the process of imaging the OMV drive, Clonezilla stopped because it said the OMV drive has bad sectors and refused to build an image of my drive (this after doing an fsck -fy and letting any errors get prepared before the image backup started, still failed during backup).

I'm currently back under OMB doing a full file backup of the system drive inside of OMV itself, though that is just doing a file copy of everything to another drive in the system. I still need a way to create a new system boot drive and get all these files/config on there.

Since Clonezilla seems to be failing, what is my next approach I can make to get my entire OMV system drive over onto a fresh SSD and remove this drive from the system completely?

I just don't want to redo all my config and reinstall all my plugins, lose all their config, etc. Even if it's as easy as creating a new OMV boot drive independently (as if I am setting up a new system), then connecting it to the live OMV system via USB and somehow copying a bunch of files from the live boot drive to the new drive while OMV is running, I'd be up for that, too.

Thanks for any help.

I noticed tonight that OMV suddenly stopped seeing one of my drives.

This is just a media server with a bunch of ext4 drives, no RAID or redundancy.

Last time I lost a drive, I tried running fscks and other things frantically to get my data back and really ended up messing up the drive bad.

I'd like to not make that same mistake again.

I checked the boot log, and it looks like OMV doesn't even see the drive at all anymore after a reboot (it didn't even try to mount it).

What is the next thing I can try? I'd just like to be able to mount the drive one last time to copy the data off to a new drive.

Thanks for any advice or things I can try that will not corrupt the drive any further!

B.

Got it. So is there still merit then, assuming I can get the dd to run all the way through without the hardware failing, to trying to image that drive to a file where I may be able to rescue some of the data? What I think has been happening is that the drive has been sort of just hard-erroring out, so my assumption is that during the dd, trying to move 3 TB of data to an image on another drive where I make space...it'll be tough to get through that dd without crashing. Worth a shot though?

Thanks so much for hanging with me. Does this tell us anything?

Keep in mind this was an ext4 filesystem, which somehow now has Debian thinking it is an ext2 filesystem (a message popped up that the filesytem type changed somewhere along the way of running fsck's).

root@debian:/home/user# sudo mount /dev/sdd1 /mnt
mount: wrong fs type, bad option, bad superblock on /dev/sdd1,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

root@debian:/home/user# dmesg | tail
[ 4771.413017] ata5.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 24
[ 4771.413018] res 51/04:00:00:00:00/00:00:00:00:00/a0 Emask 0x1 (device error)
[ 4771.413022] ata5.00: status: { DRDY ERR }
[ 4771.413024] ata5.00: error: { ABRT }
[ 4771.454173] ata5.00: configured for UDMA/133
[ 4771.454179] ata5.00: device reported invalid CHS sector 0
[ 4771.454185] ata5: EH complete
[ 7216.391972] EXT2-fs (sdd1): error: can't find an ext2 filesystem on dev sdd1.
[17557.321323] EXT2-fs (sdd1): error: ext2_check_descriptors: Block bitmap for group 128 not in group (block 0)!
[17557.321328] EXT2-fs (sdd1): group descriptors corrupted

Thanks for the reply. Since this is a 3 TB drive I don't have an empty 3TB at the moment but I'm working on finding some space or will just buy another drive.

Given all the damage I may have done already, seeing as the file system somehow downgraded to ext2, and I've been letting fsck run on -y and rewrite all kinds of blocks over and over (for hours), is it possible that I won't even have any data left in there?

If you think there's still hope, I will definitely run the dd command above and work with the image. Is there any chance that I'll run the DD and the image will be unable to be mounted? I did start the dd command and it was definitely able to read the bad drive, but I aborted it when I realized I would need 3TB free for the img file to build.

I have some more details so I am giving this a BUMP in the hopes some of you can help.

I've done a lot of tinkering with this drive, right now I have it yanked out of the OMV box and I'm trying to access it on a Debian Live CD system.

Debian sees the drive but refuses to mount it. I've run so many fsck commands on this drive. Right now on the latest fsck it seems to be getting an "error reading block" on every block and doing a force rewrite. It's up to block 610XXXXX and still going strong after hours of running. I have no idea what it's doing but suspect maybe the "force rewrite" is just basically deleting all my data.

The file system downgraded to ext2 at some point from the ext4 it was set to. Debian sees it as an ext2 file system now but still unable to mount it.

Running dumpe2fs did reveal there were a lot of superblocks still intact.

Not sure what to do next. I am not trying to save the drive long-term, just one more mount to allow me to copy as much of the data off as I can.

Oh you bet it is! Just not sure what to do with the 3 5TB drives I have now -- Had I known I was buying this a year ago I'd have capped out at 4TB drives and never went higher. My bad. This server thing is a new world but very cool. Makes me wonder why I ever bothered trying to build this OMV box with a dinky little PC!

Glad you said that, I was wondering why it was cheaper and did SAS2. Crud. Thanks again for the help. I'm debating giving the models you found a try though am worried they will only go up to 4TB with the SAS backplane which I am reading. If I have to drop a lot of coin on a new backplane board, I may just want to look for one that's compatible with the larger drives. Guess I am still a little ahead of the curve there on that. I really appreciate the advice!

Dumb question #2, I can just buy that and slide my SATA drives right in there? No extra cabling/adapter cards needed, assuming the backplane itself supports the drive size?

What do you think of this one?

http://www.ebay.com/itm/2U-Sup…e88bc3:g:dvUAAOSwyLlXoniE

That's awesome, thanks. So I just set that box to boot from my OMV drive and I'm good to go, no extra hardware needed? That's perfect.

The more issues I see with trying to use a regular PC as an OMV server, the more I realize I should maybe go about this a different way. I'm up to 32 TB of media from years of TV recordings and storage and I am a stickler for local HD video and not the cloud/netflix/etc. I like having my own files!

I would like to have a server running OpenMediaVault and that's it, with a TON of hard drives in there. Are there cheap/used servers with 12-16 bays on eBay that I can just install the drives in, boot them from an OMV drive, and go?

This would be a new world for me but I've outgrown the world of a PC case with 6 drive bays. External USB enclosures are not doing it for me and expensive. There has to be a better way. I have plenty of space to put it, I don't care how big or loud it is, just so long as it can handle large drives as they get bigger 8TB+ and that I can put 12 or 16 drives in there.

I would love some advice. I just think it's time to put the PC as an OMV box out to pasture.

I had a drive that I forgot to monitor and the sucker died on me without warning. I've been running fsck commands on it but I've made it no better or possibly worse. My system doesn't have any RAID since it is just media files which I can get back (it will just be a pain to do that).

What I'm hoping is that you fine folks can help this linux novice out on a command/method I can run to at least get this drive to mount one last time to get data from it.

Right now, OMV warns me that manual fsck needs run every time I reboot. I run it on the device (by the way, since the /dev/sdX1 changes every time I reboot, is there another way to identify which drive is the corrupt one at the command line by seeing the drive label?)

Errors I have seen:

"Group descriptor XXXX checksum is XXXXX, should be XXXXX, FIXED
Block bitmap for group XXXX is not in group.
WARNING: SERVERE DATA LOSS POSSIBLE"
relocate? (Y)"


Thousands of those.


"Superblock has an invalid journal (inode 8).
CLEARED
*** ext3 journal has bene deleted - filesystem is now ext2 only *** "


Tons of "Buffer I/O error on device sdX1, logical block XXXXXXXXXXXXX"


"Error reading block XXXXXX (attempt to read block from filesystem resulted in short read) while getting next inode from scan)."

I want this drive out of my system and in the garbage as fast as possible, however, I'd really like a shot at getting any remaining data that is on there (whatever I did not already corrupt further by running these fsck -fy -c -t commands and saying yes to everything)...

I did run a dumpe2fs | grep superblock command on the file system and there were a ton of backup superblocks it found, which may be good news and somewhere to start?

Right now, openmediavault does boot and see the drive and the label, but it does not mount the drive, and throws a wicked error when I try and mount it. I don't want to blindly try throwing fsck and other commands at the drive anymore not knowing exactly what I am doing. Any suggestions on what I can do (even by removing the physical drive and booting it into a standalone Debian live CD system or something to try and get at it)...I'll try anything.

Help?

Thank you Coda, your suggestion worked great.

No idea why, but starting fresh like you said with one drive in the enclosure, booting to BIOS, seeing the drive, shutting down, adding second drive, worked great. I did put the drives in the bottom 2 bays of the enclosure instead of the top 2 bays, not sure if that had any bearing on things, but I am now up and running in OMV with the new enclosure using the old drives.

Thanks for the help!

Think I just found out more about my problem - The BIOS of my PC (latest BIOS is installed) does not see the enclosure connected.

Anyone ever had this issue before?

I upgraded my 2 bay external USB 3.0 enclosure to a 4 bay enclosure today. I removed both physical disks from the old 2-bay box and placed them in the new 4-bay box. Same vendor. Unfortunately, it seems now that OMV can't find them in the new 4 bay enclosure. I don't see a way to add these back in via the GUI, are these ssh commands I can run from the console to return these volumes?

Ideally, I was hoping that they would just be found as before and re-integrate in with all their shares, etc, intact, but OMV must be confused about something since the enclosure changed.

I'm hoping this is really easy to do and I don't need to add a new volume and copy these from another network machine over or something.

Any assistance will be appreciated. Version is 1.19 Kralizec. AMD 5600 CPU with 4 GB memory, 6 internal sata drives and 2 in the external USB bay. Would like to get this sorted before I bump to 2.1.

I don't see the new volumes listed in Physical Disks or when I do an fdisk...But the enclosure is correctly connected to a USB port on the OMV box and I can mount/read the drives when I connect the unit to another computer.

Thanks.

More detail here (and maybe I should start a new thread).

CPU usage is spiking from 0-4% into the 80-90% during ANY file copy/rsync operation between internal drives. USB external drive does not seem to be the factor at all here. Any file move/copy operation seems extremely taxing on the OMV system.

I have my drives set up as AHCI in the bios. Anything else I could be missing here. I may have been looking at the wrong place all along. External USB just may be more susceptible to data corruption due to the way it's connected or something.

My guess is that CPU usage should not be spiking like this during an internal file copy operation?

Can I ask you one more question too, since you have been so kind.

If my CPU usage is at 0%, and when I initiate any operation to an externally connected USB drive to copy files (to/from) and CPU usage is spiking up to 80-90% while that operation is taking place, is it safe to assume that I have something wrong that is happening either with my configuration or with my hardware? Or is that a normal thing to see during a large file copy operation, for 80-90% of CPU to be used?

I am running with 4GB of RAM in an AMD A6-5400 CPU box headless, onboard graphis, all sound disabled (trying to soak out as many resources as I can).

I will note I have 6 drives connected internally as well as an external eSATA enclosure with 2 more drives in it. Not sure if that eats up overhead but I do idle at 0%.

I can do some rsyncs using the internal SATA drives if you would like a benchmark. I imagine the CPU usage during an RSNYC from sata drive to sata drive is a lot less than when doing an rsync involving the external USB drive for some reason (though I can't say why?)