RAID5 or RAIDz - which is beter?

TadeuszD · Oct 9th 2025

In my home NAS, I have two 1TB hard drives (WD RED WD10JFCX). They are currently running in RAID1 (total capacity 1TB) and contain my primary data backup.

A few days ago, I bought an identical drive in good condition and I would like to expand my storage. I plan to use these drives to build a 2TB storage with a durability comparable to the current one.

After googling I'm considering two options:

1) Rebuilding RAID1 to RAID5 with 3 drives. RAID5 is simple to use and easy to configure. However, it doesn't have any tools for checking for bit-root errors. It's also susceptible to data corruption due to "write hole".

2) Rebuilding RAID1 to modern ZFS RAIDz. Unfortunately, installing it in OMV is a bit tricky because it requires a dedicated kernel. For me, it's a risk. RAIDz has self-healing mechanisms, but large logical data corruptions (e.g., due to bad sectors) are also unrecoverable, just like in classic RAID5. Additionally (though this is apparently a myth), RAIDz requires a lot of memory and a powerful CPU. Some also claim that RAIDz's self-monitoring mechanisms cause faster drive wear (more frequent head movements in the background). RAIDz supports data compression, but this slows down read/write speeds and doesn't always take effects, depending on stored data.

Which of these statements are true and which are false? Or am I missing something else?
I'd like my choice to be well-considered...

gderf · Oct 9th 2025

Maybe consider a third option?

3) Two 1TB data drives and one 1TB Parity drive using SnapRAID.

TadeuszD · Oct 9th 2025

Quote from gderf

3) Two 1TB data drives and one 1TB Parity drive using SnapRAID.

If I understand SnapRAID correctly, it will be two drives with separate file systems. Not one larger drive...

gderf · Oct 9th 2025

Quote from TadeuszD

If I understand SnapRAID correctly, it will be two drives with separate file systems. Not one larger drive...

You can merge the drives using mergerfs to appear as a single drive.

crashtest · Oct 9th 2025

Quote from TadeuszD

RAIDz requires a lot of memory and a powerful CPU.

Really? I'm running RAIDZ, using 3ea 2TB drives creating a 4TB pool, on an older Atom Processor with 4GB of ram. (The Atom CPU is a D525 with a CPU Passmark of a meager 397.) It's working fine.

Quote from TadeuszD

RAIDz supports data compression, but this slows down read/write speeds and doesn't always take effects, depending on stored data.

These days, the CPU impact of compression overhead ranges from minimal to almost nonexistent. If you're running something better than my Atom CPU, the effect would be nearly unnoticeable. Beside that, compression only works intensively and well on text files and similar data that is not already compressed. With this being the case, it makes sense to turn on compression at the filesystem level for documents. Most media files these days are natively compressed (video - MP4 and audio MP3, etc.). Given the size of pre-compressed media files, they probably make up over 95% of a home user's storage and there's very little additional compression achieved so there's little point in turning it on. If done selectively in this way (on 5% of potential storage), compression is not an issue. If you have a reasonably performant CPU, compression would not be an issue at all.

Quote from TadeuszD

Some also claim that RAIDz's self-monitoring mechanisms cause faster drive wear (more frequent head movements in the background).

I'm not going to say that this is nonsense on the face of it (thou it seems to be). However, I seriously doubt that anyone can objectively prove it.

Most drives, even commodity drives intended for consumer use, are over designed for warranty purposes. BackBlaze uses consumer commodity drives, in data centers, in enormous storage arrays that are running 24x7. Most of them do fine,, lasting for several years as they should. Your home usage, no matter what you're doing, would be a drop in a bucket compared to what BackBlaze is doing.

________________________________________________________________________________________

- If you want the absolute best possible bitrot protection, that would found in ZFS using Zmirrors (A rough equivalent of RAID1.) Note that a pool can have more than 1 set of mirrored drives and that mirrors can be added as needed. (But they cannot be removed.)
- The next comparable for bitrot protection would be SnapRAID, in my opinion, if your storage pool is largely static. (Meaning you add files but do not do large multiple file deletes on a regular basis. SnapRAID uses existing data for Parity calculations and reconstruction.)
- Next would be ZFS in RAIDZ. RAIDZ can reconstruct some error situations using Parity but it doesn't compare to Zmirrors.

Here's a overview explanation of -> ZFS data integrity.

Finally, if you're using an SBC or storage is external and connected by USB, Firewire, etc., , hands down I'd go with SnapRAID.

BernH · Oct 9th 2025

Quote from TadeuszD

1) Rebuilding RAID1 to RAID5 with 3 drives. RAID5 is simple to use and easy to configure. However, it doesn't have any tools for checking for bit-root errors. It's also susceptible to data corruption due to "write hole".

2) Rebuilding RAID1 to modern ZFS RAIDz. Unfortunately, installing it in OMV is a bit tricky because it requires a dedicated kernel. For me, it's a risk. RAIDz has self-healing mechanisms, but large logical data corruptions (e.g., due to bad sectors) are also unrecoverable, just like in classic RAID5. Additionally (though this is apparently a myth), RAIDz requires a lot of memory and a powerful CPU. Some also claim that RAIDz's self-monitoring mechanisms cause faster drive wear (more frequent head movements in the background). RAIDz supports data compression, but this slows down read/write speeds and doesn't always take effects, depending on stored data.

My 2 cents, feel free to make your own decision though.

I have used traditional RAID for many years (around 20 or so) and have data that has been carried from single drives to a RAID5 with several RAID5 to RAID5 migrations also as I increased drive sizes and migrated across many filesystems (fat32, ntfs, ufs, xfs, etc). Some of the oldest data I have is going back to the 90's. I have never encountered bit-rot on anything. That's not to say it doesn't happen, but it is very rare. If you are concerned about your data, a backup is always required. If you encounter an error in the live data set, you restore a copy from the backup. Regardless of the RAID or RAID Z level, a backup is important. You can't rely in a RAID or RAID Z to be your safety net.

RAID Z does use available RAM for it's ARC cache, but this is dynamic and is released to applications when they need it. The bigger RAM requirement comes when using the de-dupe function.

RAID5 does have a write hole, which is the time it takes for a checksum to be calculated and written. Once again, this is small, and does not effect data that is already on the array. It only applies to data in the process of being written. The write hole becomes a problem if something interrupts the write, such as a power bump or glitch. If you are serious about your system, a UPS helps protect against that. Bad hardware or cables can also cause a problem there, but that is true for any filesystem or drive.

I currently run a RAID1 and a RAID5 both using XFS.

TadeuszD · Oct 10th 2025

Thanks to all for comments!

Quote from crashtest

Really? I'm running RAIDZ, using 3ea 2TB drives creating a 4TB pool, on an older Atom Processor with 4GB of ram. (The Atom CPU is a D525 with a CPU Passmark of a meager 397.) It's working fine.

Thanks for explanation. My CPU is also week (AMD E350 APU), so I was a little worried about the load increase too much.

Quote from crashtest

These days, the CPU impact of compression overhead ranges from minimal to almost nonexistent.

Thanks for this information. My storage contains everything you can find in typical user's home directory (documents, config files, even executables) except multimedia. For large multimedia archives (photos, movies) I have dedicated storage (uncompressed). That's why I'm thinking about turning on compression on my new backup storage. According to OMV ZFS plugin documentation lz4 should be good enough.

Quote from crashtest

- If you want the absolute best possible bitrot protection....

Quote from BernH

I have used traditional RAID for many years (around 20 or so) [...] I have never encountered bit-rot on anything.

A few words of explanation.

During my entire adventure with computers (I also have archives from the 90's) I have never experienced problems related to so-called "bit-rot". I've seen a lot of damaged drives, both mechanically and electrically (broken heads, scratched platters, burnt controllers), but I've never met such random errors. As long the drive is in good condition, these errors should be corrected "on the fly" by the disk firmware using CRC.

However, everyone around is now writing about it as a troublesome plague.

That's why I started to investigating this problem. But personally, I'm not afraid of this issues more than complete disk failure.

Edit:

This is very interesting article about "silent" (bit-rot) data corruption:

What home NAS builders should understand about silent data corruption

According this article using Non-ECC RAM modules is more dangerous for data integrity than disk read failures.

crashtest · Oct 10th 2025

Quote from TadeuszD

My CPU is also week (AMD E350 APU), so I was a little worried about the load increase too much.

You E350 is more powerful that my Atom CPU. It's not going to break any speed records but it will work fine as a NAS in a file-server role, with ZFS, if you want to go that route. Just don't activate deduplication (which is OFF by default) and you should be good to go.

Quote from TadeuszD

lz4 should be good enough.

LZ4 is a good, overall, compression algorithm with, maybe, a 1 to 2% CPU performance hit. (It's unlikely that you'd notice it. I don't)

Quote from TadeuszD

I have never experienced problems related to so-called "bit-rot".

Actually, you don't know this for a fact. I had old files going back to the late 80's and, at one point in the late 90's, I lost irreplaceable files to a rapidly failing hard drive. (This was an older drive, during an era where SMART was still being rolled out.) That was my lesson learned.

A "flipped bit" (bit-rot) could translate to nothing more than an artifact (a single pixel changing color) in a photograph. In a document, it could mean a misspelled word that crops up in a 10K word essay. Where the problem actually lays is with the various failure modes of hard drives. Most people think they fail and die like a light switch, on/off. They usually fail slowly and quietly, with randomly failing magnetic media. Drives do have some countermeasures for this kind of thing (marking bad sectors) but fixing file error conditions is not guaranteed or even reasonably certain. Further, most filesystems are capable of silently inserting errors as well, during write or rewrite operations. In the case of traditional RAID, the RAID layer is very capable of inserting errors during array writes that the file system would be completely unaware of.

Another thing that is often ignored but has a much greater impact is "malware" and/or the malicious encryption of user files. ZFS overcomes these issues with "Copy On Write" and snapshots. First, all files written are check-summed. (In this day and time, I believe all filesystems should do this but they don't.) If a file is modified in any way, a full new copy of the file is written and, depending on the configuration (snapshots), the old original copy is retained. These features can be used to completely defeat encrypting malware, misbehaving hard drives and so called "cosmic-rays" that randomly flip magnetic media bits.

Quote from TadeuszD

According this article using Non-ECC RAM modules is more dangerous for data integrity than disk read failures.

While I use ECC, I don't see it as an absolute necessity. Most home users are going to save a file and "done". If the file is in the ZFS array, it's checksummed and protected. Most file operations, from there, are simple reads. As long as this is the case, the file is continually protected. The only way a given file can be corrupted by RAM is if it's read into memory, modified (then randomly corrupted) and, finally, written back to the array. Can corruption happen like this? It can, but I tend to believe that it would be fairly rare.

In any case, much as I believe ZFS is a good choice for protecting data, I believe the same with regard to ECC. I use both.

TadeuszD · Oct 11th 2025

Quote from crashtest

In any case, much as I believe ZFS is a good choice for protecting data, I believe the same with regard to ECC. I use both.

I read a few more articles and forum threads about ZFS and SnapRAID/mergeFS.

I think that snapRAID/mergeFS is better for "static" archives (ie. photos and movies). But for frequently and randomly changed data (backups of working folders) ZFS will be better solution. So, I decided to convert my current RAID1 to ZFS pool with single RAIDz vdev.

I don't plan to add new VDEVs to my pool. I assume that in the future (if possible) I will only replace the drives with larger ones and increase the VDEV size.

I started to install Proxmox kernel and I have first question:

My OMV uses 6.12.43 kernel. But Proxmox kernels are available only for versions 6.8, 6.11 or 6.14.

If I understand correctly this post: https://forum.proxmox.com/thre…t-no-subscription.164497/ the "default" Proxmox kernel is still 6.8. 6.14 is only optional.

Should I take a step back and use Proxmox kermel 6.8?

crashtest · Oct 12th 2025

This -> doc will explain the kernel situation along with a few other ZFS consideratons when installing ZFS on OMV.

For instance, you can add more VDEV's as needed but, once added, you can't remove one of them. (In your case it would be best to add a like VDEV of RAIDZ but it is possible to add a mirror.)

While a RAIDZ VDEV will scrub for errors, if you want the ultimate in bit-rot protection, for important documents or something else, use copies=2 on the filesystem they're stored in. The doc explains that as well.

Quote from TadeuszD

My OMV uses 6.12.43 kernel.

I'm guessing 6.12.43 was a choice from the Kernel Plugin? If so, you should be fine.

Quote from TadeuszD

Should I take a step back and use Proxmox kermel 6.8?

You could if you wanted but, if you already have 6.12 installed and it's working well, I'd go with it. It's in the middle of what's available so it's a reasonably safe choice. I never do bleeding edge kernels in any case. Their userland package sets may have issues, may be missing packages or be generally unsettled. When it comes to servers, I like "conservative". Bleeding edge kernels are for bleeding edge hardware. If you hardware is 2 or 3 years old, they're not needed.

TadeuszD · Oct 12th 2025

Quote from crashtest

This -> doc will explain the kernel situation along with a few other ZFS consideratons when installing ZFS on OMV.

I spent half the night testing various configurations. Then I decided to use Proxmox 6.14 kernel. I feel braver because ryecoaaron also uses this kernel version.

Quote from crashtest

While a RAIDZ VDEV will scrub for errors, if you want the ultimate in bit-rot protection, for important documents or something else, use copies=2 on the filesystem they're stored in. The doc explains that as well.

Yes, I know it. But I suppose that copies=2 will consume 100% more space than copies=1. My disks are too small for this kind of extravagance. I also use rsync and borgbackup to create additional copies of my of backup. I'm not sure I need one more level of redundancy...

For my purposes I created one pool with one vdev with one filesystem:

I carefully studied zfs man page: https://openzfs.github.io/openzfs-docs/man/v2.4/index.html , but I used default values in most cases:

ashift=12,

recordsize=128K

I decided to control compression on filesystem level. For pool compression=off, but for backup filesystem compression=lz4.

I hope that this configuration will be good enaugh.

TadeuszD · Oct 13th 2025

Time to tests...

I measured write speed for my new ZFS storage during copying 3 data sets:

174033 MiB / 33min. -> 87,9 Mib/s (typical office files, pictures)

217274 MiB / 52min. -> 69,6 MiB/s (bunch of small files, source repositories)

152541 MiB / 28min. -> 90,8 MiB/s (large files, i.e. VM and snapshots)

Everage speed for all data sets is 80,2 MiB/s. Everage compression ratio is 1,24. Everage CPU load during copying files was 90-95%.

I'm not so happy with this numbers. On 1GB LAN (~100 MiB/s) the CPU/ZFS storage will be the bottleneck...

BernH · Oct 13th 2025

Quote from TadeuszD

This is very interesting article about "silent" (bit-rot) data corruption:
https://louwrentius.com/what-h…lent-data-corruption.html
According this article using Non-ECC RAM modules is more dangerous for data integrity than disk read failures.

Once again, I personally feel this is all blown out of proportion. If non-ecc ram were such a horrible thing, it would not be used at all. ECC ram does help correct against errors as data passes through it, but as with the whole bit-rot thing, errors directly related to the ram are extremely rare unless you have bad ram, but that would likely be causing more problems that an occasional bit flip.

These fear mongering articles about bit-rot and ecc vs non-ecc ram, do have a place when you are talking about extremely critical data, such as that held by your bank, your health care system, your social security system, etc. but when it comes to a normal home user's nas, 99% of the data is not that critical, and the little bit that is should be protected in backup.

crashtest · Oct 14th 2025

Quote from TadeuszD

I'm not so happy with this numbers. On 1GB LAN (~100 MiB/s) the CPU/ZFS storage will be the bottleneck...

Note there's a difference between mega-bites and mega-bytes.

In any case, you don't have screaming fast hardware. If performance is the ultimate goal, a AMD E350 APU is not going to give you what you want. On the other hand, once your data is stored, I'm sure the AMD CPU will be able to serve or stream all the files your users (at home) might need.

TadeuszD · Oct 15th 2025

Quote from BernH

These fear mongering articles about bit-rot and ecc vs non-ecc ram, do have a place when you are talking about extremely critical data, such as that held by your bank, your health care system, your social security system, etc. but when it comes to a normal home user's nas, 99% of the data is not that critical....

Yes, you right, we are not building a systems critical for a mission to Mars. But still, if I can secure my data better, why shouldn't I? Classic RAID systems seem a bit outdated today. They have not changed significantly for 40 years. ZFS looks like a modern, stable and safe replacement even if it consumes more CPU/RAM resources.

I'm not talking about backup. A backup is necessary, but a good backup is one that will never need to be used.

Quote from crashtest

If performance is the ultimate goal, a AMD E350 APU is not going to give you what you want. On the other hand, once your data is stored, I'm sure the AMD CPU will be able to serve or stream all the files your users (at home) might need.

Performance degradation is not a key issue. This is a price I am able to accept. A NAS is not a gaming PC.

By the way, yesterday I did a "crash-test" of my system. I simulated the loss of the system drive and/or motherboard. I moved the ZFS storage to freshly installed OMV instance. The new system imported the storage correctly (I only had to use the "force" option). The "Discover" function correctly detected the file systems and mounted them in the previous mounting point.

Now I'm sure that in case of failure, I will be able to restore the system.

BernH · Oct 16th 2025

Quote from TadeuszD

Yes, you right, we are not building a systems critical for a mission to Mars. But still, if I can secure my data better, why shouldn't I? Classic RAID systems seem a bit outdated today. They have not changed significantly for 40 years. ZFS looks like a modern, stable and safe replacement even if it consumes more CPU/RAM resources.
I'm not talking about backup. A backup is necessary, but a good backup is one that will never need to be used.

There is nothing wrong with ZFS, but there's nothing wrong with a classic type of RAID either. If there was, they wouldn't still be used. For example, at the office we have a SAN system (not a NAS) that is designed for high performance iscsi connections for our whole facility to work from for video and audio editing and finishing. This system is comprised of 4 chassis, each containing 16 seagate x18 Exos HDD's configured in a large RAID 60 using XFS as the filesystem where each HDD chassis has 2 RAID 6 arrays with a RAID 0 on top of them all, and another chassis with 24 SSD RAID 60 in it that acts as a cache tier. It is configured like this from the manufacturer for drive redundancy, speed, and parallel access from multiple workstations.

As for the outdated comment, if you are a computer history buff, you will also know that zfs is not new. It is actually about 25 years old in it's own right (created by Sun Microsystems), so age of a technology does not make it bad. The BTRFS filesystem is much newer than ZFS, but it doesn't mean its better because it's newer. The XFS filesystem I use (created by Silicon Graphics) is about 8 years older than ZFS, but as with my example, it is far from outdated or dead and is still being maintained and developed.

When I built my current system, I was contemplating switching to zfs, but at the time there was a zfs docker bug that, along with zfs not being kernel native, made me opt to stick with my tried and true RAID 5 with XFS configuration. Next time around I will refresh myself with the current state of ZFS and decide again at that time if I want to use it, but the kernel native issue is one that is important to me.

RAID5 or RAIDz - which is beter?

Participate now!

Tags