Create a degraded RAID 1 to add a 2nd disk later

Hello everyone!

Problem
OMV's interface won't allow creating a single disk RAID 1 array, it asks for minimum two disks.

Question(s)

So I'm wondering: Can I possibly create a degraded RAID 1 array by hand, but still have it managed in OMV's interface?
What would be the best way to do that in your opinion?

Please, note that I'm a bit new with RAID and relied greatly on OMV's interface until now. I've seen actual documentation about RAID and mdadm commands but never used it by hand. Therefore, you shall not hesitate to point out actual relevant documentation in this case or useful commands, that will be much helpful and appreciated.

---------

If you want more details to answer and appreciate the situation:
Context

Goal

.

Thanks a lot to anyone taking time to help!

I'm not THAT new with the idea of RAID and all the theory about it. I'm passionate about IT and I'm also a Linux sysadmin...
I've just not been handling the server installation and RAID rebuilding part myself, which is why I'm weirdly experienced with Linux (enough for some to call me an expert) but just not that much with RAID. I'm also more comfortable with resizing virtual disk images and file systems than working with physical drives and mdadm.

Also, I'm not affraid of creating the RAID myself with CLI, I already found many doc about that and it's not complicated at all, the question is rather about OMV: before I do that on my prod NAS (which I need working since it hosts important stuff that I need soon), is there any possible issue with OMV if I do that? Is there a config file to alter, or anything like that?

To address your observations:

I know you see people using RAID as a backup all the time (hence your signature, @geaves), therefore you assume I do think it is a backup. But please, re-read my sentence: Like I said: "adding a little bit more safety to this data". It doesn't mean "backuping my data in RAID 1" which would be absolutely wrong.
I know RAID is not a backup, thanks. Also, this is non critical data and I won't pay 750€ (literally the price of adding 3x8TB drives) to back it up. It's not worth it.
I know the risks and that's not the point of my topic.

Ultimately, RAID 1 is NOT useless and you should in my opinion never tell that to a user @tkaiser (especially if you think he's a complete newbie). That's a total extremist point of view which is totally wrong: It's of course not the safest, but it's also perfectly enough in the vast majority of disks failures.
It's for sure almost 2 times safer than no RAID at all and it's more affordable than RAID 10 (which costs twice as much, for a reminder, cost is also important for normal human beings, even companies).
I've been managing servers since a few years, in a company using only in RAID 1 servers.
I've been working closely with my team, swapping disks myself in the datacenter, and I know for sure it allowed replacing many -and actually all- failing disks without any data loss or need to grab a backup.
So even though RAID 1 doesn't have a permanent checksum/consistency check, the event of having both drives writing garbage at the same time is so unlikely, especially, if you monitor SMART errors (which we do and OMV allows to do and I use + I check it manually from time to times), that I've never seen it happen. Therefore, RAID 1 is not useless since my experience proved it is working perfectly and is suitable even for server application.

Now this has been clarified, can we get back to the topic?

Thanks

Since many users (including myself indirectly) showed success with RAID 1, then it is useful, and therefore it cannot be "useless".
I cannot agree with that wording since it goes against logic.
If you have relevant and common examples where RAID 1 does not work or is useless (which I doubt they would be that common since I've never experienced them), then please provide them so that I can maybe understand a little bit more about your point of view.

Thinking that there are better options is totally acceptable, but it doesn't make other options immediately useless.
However, If these options are so much better than traditional mdadm RAID, why don't we see those in OMV when going to the RAID section? Or not even a warning?

That said, I'm 100% willing to learn more details about these options, and if you have a better proposition to make that would suit my case scenario (1 disk with non critical data, that will get later any kind of automated and low performance cost redundancy to protect against a disk failure), then you are 100% welcome mate.

Edit:
PS1: I didn't see the link I'm reading atm.
PS2: I'm not doing business work at home, but that's still work and I don't want downtimes which is why I'm interested in RAID. Also, that's a way to have a better data freshness in the event of a disk failure: You usually don't need to recall a backup.
PS3: I mentioned RAID 10 because It's known to have more safety.

EDIT: Crap, you answered in the meantime while I was writing!
Reading your answer!

So, I've read your post.

"RAID is only about availability (who needs this at home?)"
Well, I do.
One single good reason (but I have way more since my NAS is heavily multi-purpose) is that I'm running daily backup of my web server on my OMV NAS. I don't want this backup interrupted and I also don't want to lose these in the event of a single disk failure. Also it would be unreasonable to spend more money on other drives to backup this backup.

"It allows to continue operation without interruption if a single disk COMPLETELY fails. It doesn't deal good or at all with other failures that happen way more often."
It also allows to continue operation if a drive just starts failing and writes garbage: you just eject it, replace it, rebuild and voilà.
Speaking of that, I shall mention I've tried the operation with my RAID 5 before putting it in production, and OMV handled that perfectly.

"with cabling/connector problems that are undetecte there's a chance that different data is written to the 2 disks"
Well, who cares? If that happened to you, it's the same as a failed drive, you remove it from the RAID and rebuild.

"Since mdraid only reads from one disk (not checking for mismatches)"
Well, it doesn't do on the fly, but you can still check the RAID, which will find mismatches. By default, there's a monthly cron for this. If not detected upon writing or with a SMART error, then the cron will help. Even in the worst case scenario if you waited one month to discover that, chances for the other drive to write garbage as well are insanely low.

I'm not sure if you've been traumatized by a bad experience with RAID 1 or something, but to me it seems like you are underrating this RAID more than what I would consider reasonable. I don't see any valid argument against RAID 1 here. What would be a valid argument would be an alternative that offer the same capabilities which I've not come across yet.

I've also read the link about RAID 10, though the article is really poorly structured and incomprehensible.
What I get from it is: if using 4 partitions, then you're unsure if they're spread correctly for redundancy.
With mechanical drives: it degrades performance, so that's not viable.
If running 2 straight drives in RAID 10, as you would make a RAID 1, then I've only got questions.
It's written everywhere that the minimum requirement for RAID 10 is 4 drives. So can you even create it? Then you only have mirror and no stripping, so what's the difference compared to RAID1? Is it considered a degraded RAID 10, or normally operating?
Any enlightenment appreciated on this.

@tkaiser Unfortunately the "RAID-1 vs. data integrity" link doesn't work for me (access denied).

I get what you dislike about RAID 1, but it still doesn't make it worthless in most cases. I've never seen these described scenarios in practice. Unless the user is totally careless for months and has no monitoring while using cheap ass SATA controller and/or cables, and never looks his SMART, one will notice a SATA/BUS/drive or whatever error could happen before it's too late by just seing a SMART error or something.
IMO, a better approach than "mdraid1 is useless" would likely be "hey, with ZFS you get checksums/data integrity checks, you should have a look, here is some documentation".

I've seen upon looking info about OMV5 that maybe OMV6 will only support ZFS and such, breaking compatibility with existing formats?
While it's probably a good idea to promote newer (and likely better) systems, it would benefit from very good documentation and some cohabitation with existing filesystems and RAID, awaiting for people to migrate their data to such file systems. I don't see ext4 being abandoned that fast.

"(rm -rf is a good illustration why RAID is 100% useless when we're talking about data safety)."
Well, you can technically recover files from that. I've already done such recovery for a virtual machine that had no backups after a human mistake. The VM had an ext4 image on a ext4 host partition in a RAID 1
You can always blame the user for not having a backup, but shit happens to everyone, and I'm not sure if I would have been able to find a tool to recover deleted files from a directory with another file system. That time, it saved the day for the customer to be in ext4 and that a tool existed.

"mdraid users here in the forum complain that all their data has gone every other day..."
> Well, I feel you, in LinuxGSM users complain that their linux user cannot write files because they created directories as root and crap like that all the time, or that they cannot download the files but don't have curl or wget installed... That's why good documentation is important, that way the user can find and read it, and if they didn't then you can just send the link instead of getting mad every time (speaking from experience)
For my part, I am very careful with my RAID, also I have coworkers I can ask for help in the event of an issue, so I'm not too worried about this case scenario. I'm also planning a full deported backup server for important data redundancy and better automated backup in the near future.

But you forgot a huge good point for RAID 1: if one does crap with one disk in RAID 1, then they still have a second chance to not mess up with the second one!

I think I start getting the advantages of ZFS/BTRFS but you're right, I don't have any idea of how it works in real world and it will require some new knowledge and tools. Any documentation or relevant link would be welcome.
Do OMV 4 (or the 5 beta) offer any kind of tools to help managing these? I mean, you can create these file systems, but especially, is there anything to help with redundancy on these like the great mdadm integration provided?

Like you said, the level of trust is still not very high especially for BTRFS and all the bugs and problems that have existed. Even if most are solved (not for RAID 5, it seems), the scars still exist and people are not ready to consider it seriously.
And ultimately, people (and me included) prefer working with stuff they're more at ease with and with proper documentation if needed.

About RAID 10:
Some datacenter dude told me once there was some kind of possible checksum with RAID 10, maybe he was wrong or maybe he was talking about a different RAID or filesystem and I can't recall.
Since you know RAID better than me, maybe you might see what he could have mentioned? RAID 5/6 maybe, since they have some parity checks, are they more suitable for data integrity?

In the end, back to the actual goal of this topic: you know my case well now: do you have a better solution for my 8TB drive than the one I came up with in this topic, to have some kind of local automated redundancy, without performance loss, that I'm not forced to setup right now, and that won't require emptying the drive first?

Thanks

Edit:
Sorry for the loooong posts, that's the price of interesting developed conversations, I guess.

Zitat von tkaiser

You're confusing RAID5 and RAID1 here. What I mentioned only applied to RAID1 (that's the RAID mode I call 'almost useless').
With RAID5 there's parity information and as such corrupted data could be reconstructed (but you need to fear the 'parity RAID write hole' in such situations and rebuilds take ages that's why RAID5 with modern TB drives is insane as well).

With mdraid's RAID1 there's nothing. And that's the real problem with wasting disks for an mdraid1: no data integrity protection and not even checking. You simply don't know whether a failure somewhere rendered your data corrupt. And since better alternatives like a zmirror or btrfs RAID1 exist, IMO we should really stop to waste disks for almost nothing.

So in RAID 1, if there is an mdadm check, and a drive wrote garbage, you're saying it won't be detected?
My understanding is that mdadm would show an error, and you can then easily identify the failed drive by reading the SMART attributes of both drives and finding the one with unrecoverable errors or failed sectors.
Therefore, if I'm right, then there is still a way to recover from a failing drive in mdadm RAID 1 without losing anything or require a backup restore.
OK, the detection is just not at file system level, and a bit delayed since you need to wait for the next check (1 week to 1 month usually), but it still works.

About rebuild time, for reference, my 3x4TB RAID 5 (two ST4000DM000-1F21 and one new ST4000VN008-2DRA) took around 8 hours to rebuild (same duration as the initialization since I wasn't really using the array upon rebuilding).
That's about the time required to write 4TB at an average of 140MB/s.
For me, that's fast enough, but it is true it could have been faster if only actually used space was written back as it seems to be with other RAID methods.

Zitat von tkaiser

I don't think so. On x86 I would most probably go with a zmirror while on ARM due to different kernel support situation I would use a btrfs RAID1 and then create snapshots regularly (snapshot as in 'almost backup'). The respective tools aren't integrated in OMV at all: https://www.znapzend.org (ZFS) or https://digint.ch/btrbk/ (btrfs)

I'm in x86_64 here, so I would rather go with ZFS/Zmirror. However since it's not implemented in OMV and want to keep things simple, I'm unsure to be ready for that just yet. Probably a next step in my geek life but for now I don't have the time and energy to really dig into it enough. Thanks for pointing it out though, and offering new perspectives, it will for sure be useful at some point!

Zitat von tkaiser

Exactly. Since it's impossible due to the whole concept of data integrity being unknown to primitive RAID-1 variants like mdraid's. If you don't believe me maybe you believe the authors: https://raid.wiki.kernel.org/index.php/Scrubbing_the_drives -- the difference between parity raid and a primitive mirror is well explained there.

Thanks for your answer and for pointing out this useful doc. I for sure have a lot to learn about RAID, I'm just really digging into it.

I've just learned on that wiki that you can convert RAID 1 to RAID 5 https://raid.wiki.kernel.org/i…ror_raid_to_a_parity_raid
That's kind of revolutionary info to me. :o))

I assume the interesting part for our discussion is the following:

Zitat von raid wiki

If the array is a mirror, as it can't calculate the correct data, it will take the data from the first (available) drive and write it back to the dodgy drive.

So as I understand, if it finds out there is a dodgy drive it's either because it couldn't write on it in the first place, or that there are 3 drives or more in the mirror and it found one with different data on one of them.
The real question is: if there is no distinction between drives's sanity and it cannot assume which is the dodgy drive (example 2 drives RAID 1), then what does it do?
Does it say "RAID status=fail", then human interaction is required and you check the SMART and quickly find out which drive to replace; or does it blindly consider the first disk of the array to be accurate and possibly writes garbage data the the sane disk?
Since I've never seen it happen, I assume with some confidence that it has near no chance of happening: that's my point that would need to be invalidated so that I can change my mind.

It might be my near Asperger syndrome, but I find this wiki lacks a bit of accuracy to prevent any doubt like that.

They also say:

Zitat von raid wiki

Drives are designed to handle this - if necessary the disk sector will be re-allocated and moved. SMART should be used to track how many sectors have been moved, as this can be a sign of a failing disk that will need replacing.

To reformulate, upon data rewrite, SMART will help knowing if there are any errors. In fact, in practice, there should be an error in SMART since the first error (unrecoverable error). Unless maybe it was writable, but then it doesn't read as expected afterwards, therefore mdadm detects an inconsistency; if so, there should also be other SMART errors on this drive, because a drive usually starts to fail globally, not on one sector.
That is my whole point since the beginning:
SMART data is usually accurate and quickly alarming enough if don't buy poorly made drives. Therefore, if you monitor for SMART errors, you'll find out that a drive is failing before even having a RAID check, and therefore, garbage won't be written to your whole array.

Zitat von tkaiser

Using single redundancy with those drive sizes and such commodity hardware is a bit risky and once you add more redundancy (RAID6) rebuilds take even longer. And as soon as the RAID will be used while rebuilding the times needed explode: https://blog.shi.com/hardware/best-raid-configuration-no-raid-configuration

Well, they don't detail their protocol well enough, but seriously, 1.5 hours to rebuild at idle but 134 hours to rebuild at load? That's almost 100 times longer, that's unrealistic on well suited servers...
That kind of delay would mean your I/O would be 100% saturated before the rebuild and during the whole rebuild time which basically means there is a problem on your server in the first place, and you should resolve it first.
I do server management, so I would detect such a loads before it's too late and advise anyone to change hardware (go for SSD, add RAM to favor caching and prevent swapping, spread the load across multiple servers or add a drive array to spread I/O or whatever) before reaching such a critical state, and I hope any sysadmin on his own would do the same.

Also, I don't see how they came up with 77h rebuild time for a 10TB drives array... If I compare to my numbers (8 hours on 4TB drives): Proportionally to the disk size, it should take around 20h. And since a 10TB drive should be faster than a 4TB drive, it should be even faster than that.

In the end, author's main point is "it takes too long to rebuild": I can say with a pretty high confidence it's not a problem unless you're over-using your I/O in the first place.

Now their solutions:
- Application redundancy: Nope. Not nearly as convenient as RAID, also the cost is twice the server price... I don't know any business that would go for that solution at the moment. A daily backup or more often + RAID 1 is the solution of choice at the moment.
- Erasure coding/Software defined storage: Requirements are likely met by ZFS/BTRFS RAIDs, so that's valid.
- Solid State Storage: Well, that was my point since the beginning: If your I/O is saturated, there is one obvious solution... Get a storage that multiplies I/O speeds by 4 to 50.

Zitat von tkaiser

There is a ZFS plugin and creating a zmirror is really easy. And by just using a storage implementation that has been developed in this century (ZFS/btrfs) and not the last (mdraid/RAID1) you get the following for free:

• bitrot protection
• flexible dataset/subvolume handling overcoming partition limitations
• transparent filesystem compression
• snapshots (freezing a filesystem in a consistent state)
• ability to efficiently send snapshots to another disk/host (backup functionality)
• immune to power failures due to CoW (Copy on Write)

(though the latter requires disks with correct write barrier semantics -- if not you need an UPS -- and large arrays require server grade hardware in general)

Found this: https://docs.oracle.com/cd/E19…819-5461/gaynr/index.html

There is also the ability to create cache devices which I had heard about (likely in LinusTechTips Petabyte project or something like that), that's cool. Hopefully you can add cache later (I assume you can, otherwise it would be a bummer).

ZFS seems very interesting for sure, I am 100% convinced. I will dig into it at work and run experiments. If it shows to be stable, understandable and "monitorable" and everyone is convinced, then we might use it for new servers.

As for my home use with ZFS, I would have two questions before going for it:
Could I create a pool of 1 drive, then add a second one for duplication later, without the need of moving the whole data?
Also, is it possible to convert a ZFS RAID 1 to RAID 5 (or equivalent) like with mdadm?

Thank you