You're not gonna believe me. You're gonna say my hard drives failed. My hard drives have NOT failed.
I have OMV running in XCP-ng (hypervisor). XCP-ng is passing the removable disks directly to OMV. 4 of my drives are working fine, but earlier this week, one of my 2TB drives (2TB_1) started reporting IO errors, and then appeared to fail entirely. I have a daily sync of this drive to 2TB_2, so no harm. I just disconnected 2TB_1, changed some paths in OMV and bought a new 2TB drive. My file shares were back up and running in no time.
Last night when I connected the new 2TB drive to OMV, all hell broke loose. 2TB_2 AND the brand new drive both begin spitting IO errors. I begin to think it's an issue with my docking station. I move both drives to a different docking station. I try one in the first, one in the 2nd. I try the drives in every configuration I can think of. I try attaching the drives before OMV is powered on, I try adding them after OMV has booted. No matter what I do, IO errors. I tried connecting only the brand new drive and formatting it. IO errors. In fact, it has even gotten to a point where OMV refuses to boot if I connect 2TB_2 while the VM is powered off. I often see the IO errors begin to pop up more frequently when I access the Encryption tab in OMV. And I've also noticed that 2TB_2 now no longer appears in this list, telling me that OMV is having read errors while trying to determine if the drive is LUKS encrypted.
So I take the brand new drive and 2TB_2 and connect them to my lubuntu laptop. Both drives work perfectly. I format the new drive as EXT4 with LUKS and rsync the entire 1.6TB of data from 2TB_2 to the new drive with ZERO errors. So now at least the data is definitely secure. However, I still get IO errors upon connecting either of them to OMV. The IO errors are always at block zero and a very high block number (possibly the last block). I've also plugged in the failed hard drive to my lubuntu laptop and am not convinced yet that this drive failed.
I know that none of this is related to XCP-ng, as XCP-ng is simply passing the unmounted drives directly to the VM, and is not reporting any errors at all. All errors are coming from OMV.I'm unsure what to do next. At this point, I think there is something wrong with my OMV VM. I have the Backup extension from OMV-Extras installed, with some recent backups (dd mode), but I'm realizing that I have absolutely no idea how to restore from these. I was considering simply spinning up a new OMV instance and starting from scratch.
Does anybody have any ideas or suggestions?