Use case RAID5 on LVM mixed HD sizes

No. If the 1.5TB drive fails all your data is lost, for sure. Use something like Greyhole or manual backup with this kind of hard drive setup.

Greetings
David

What you are looking for is greyhole. It supports drives with different sizes. I believe there is a cli howto on the forum.
Only use drives of same type and capacity in raid soft or hard. If you want to use LVM do raid first then LVM on-top of raid. That is my current setup. I have 12 drives of 3 terabyte each and have 3 software raid 5 of 4 drives each then I have LVM on top of each raid 5 setup. LVM is just a container so this way I can add another raid setup and continue the from another raid partition if that gets full.

And I still don't get why I should use LVM ontop of a raid... at least in a SOHO environment... why use "partitions" at all?

Greetings
David

Its a personal preference. Some like LVM some don't. I like it because it can manage lots of raid partitions in one central place. Its just a large container that holds all your raid setup. It works great in hardware raid or software raid and it is very stable.

Ah okay. If you like to do so then its okay. I (may) think in bigger dimensions and just add another disk to my array and grow the filesystem or add another array with bigger hard disks the next time, I have to grow. Still 3TB to fill, should last at least last till Summer...

Greetings
David

The reason I mainly keep 4 drives in a raid 5 setup and not have more drives is that all my large drives are WD Red drives and WD does not recommend more than 5 drives to be used in a raid setup. WD red are consumer grade drives with a modified firmware that can be used in hardware raid setup but they put a catch that more than 5 drives are not recommended. They don't want the red drives to out sell their enterprise drives which are double the cost. I even spoke with WD to confirm this. So I don't take any chances and have all my drives at 4 drives per raid setup. Then I can grow as I need with LVM management. Why I choose the WD drives is best bang for the buck even though they are not the fastest but they run cool and use just under 5 watts per drives compared to 8 to 10 watts for others.

Another reason is raid 5 has only 1 fault tolerance per array. The more drives you have the more of a chance that 2 drives will fail at the exact time. However keeping the array small like 3 to 4 drives has a lesser chance that 2 drives will fail at the same time compared to a 16 or 24 drive array. I choose 4 drives because I use a norco case which has 20 drives. Each row holds 4 drives so its very easy to manage. Every row is a raid 5 array then manage all the arrays with LVM.

Could you link me the statement that WD doesn't recommend more than 5 drives in an Array? Also 4 drives in a raid 5 does result in bad write performance, 3, 5 or 9 disks are best for writing.
REDs are enterprise class, but are cut in the featureset. They are designened to run 24/7 and have extended RAID controller support, all you need. Double the price? Nah, RED 3/4TB 109/157€ BLACK 3/4TB 158€/200€ ... Also the BLACKs offer 5 years warranty against 3 on the REDs.
Your case supports 20 drives? I would suggest to use 4 Arrays with 5 disks then

Well, I wouldn't risk the use of LVM just because its makes one big filesystem... I would rather split my files over the arrays and put specific files to each array.

Bigger arrays have a higher risk of fault rate? Have a look at http://blog.backblaze.com/2013…brations-storage-pod-3-0/

Also maybe worth reading: http://blog.backblaze.com/2013…long-do-disk-drives-last/

Greetings
David

Here is PDF directly from WD http://www.wdc.com/wdproducts/…Sheet/ENG/2879-771442.pdf. WD drives are designed to run 24/7 however they are consumer drives not data center drives. They are build to run a bit cooler due to the fact that many people use them in DVR CCTV setups and small nas cases but they are not designed to run in large raid setups. LVM is just a container that holds the info of where the partions start and end nothing more the data protection is handled by the raid soft or hard. LVM gives the flexability of growing a partition very easy. The code base is very stable and even all vituralization centers all use it for their back end systems. If its good enough for them its good enough for me. I would not use WD reds in a large array setup as they are not designed for this purpose. As for the 4 drive setup I'm not to concerned about the speed because its a storage nas . If I want speed and data protection I would use 8 disk raid 10 which would give 8 times the read and 4 times the write and use SSD instead of platter disks. Raid 5 is cheapest method with some data protection and maximum disk space.

They are 3 directories to backup if you use LVM
/etc/lvm
/etc/mdadm
/etc/fstab
If you are using hardware raid then no need for /etc/mdadm but /etc/lvm is required. You can get away with not having /etc/fstab because you can get the uuid info from /etc/lvm. I do daily backups to USB of all 3 dir to USB stick with the USB backup plugin. It works great. I'm still using software raid in current setup at about 270 megs per second. Not that bad for software raid.

In my nas setup I do 80 % read and 20 % write so I not to worried about the 4 disk issue. Like I said its just easier to remember each row in my case is a raid 5 setup. I have a chart that has every serial number so if a disk goes bad I now which one it is. So far its been just over 1 year and no disks have failed. In my older setup I had 2 raid 5 setups in hardware raid running for just over 5 years without any hardware failures. The best you can do for platter disks is to have a good quality power supply and make sure the disks run as cool as possible. In my old case I had 13 drivers in a tower and needed the side cover removed and in the summer I also ran a fan to keep the drives cool. Heat is the #1 killer for platter drivers. The #2 killer is poor air circulation what they call dead air you need air flow without it leads to heat and the #1 reason why drives die. Also in my old setup all 13 drives used the 5 1/4 to 3 1/2 adapters so their was spacing for air flow between each drive. I never install a hard disk in a 3 1/2 slot unless its the only disk in the case. If you got lots of disks best to use rackmount 4U cases as they are designed with proper air flow in mind compared to most tower cases.

You will also kill your drives faster (it may be minimal, but you do more "harm" to your disks) with a 4 disk raid5 then a 5 disk raid5 array, because you have to read and rewrite blocks again on multiple writes...

"WD Red NAS hard drives are recommended for use in home and small office 1-5 bay NAS systems. For systems that use more than 5 bays or are rack mount solutions, please consider WD Datacenter hard drives.*" Doesnt makes sense why more than 5 disks in a single array would concern the disks... just sounds like marketing to protect the BLACK line...

Of course, you're right that raid5 is the cheapest method to get some protection on big data stores. But lets be honest, we love it for that. But I consider while growing my raid the next time to make the step to raid6, depending on how cheap the 4tbs are when I have to buy new storage...

270megs? nice, could more nearly saturate three times gbit lan... Still need to get some equipment for bonding, I get a theoretical speed of over 500MB/s read and write (yeah i know, its theoretical as you loose much speed if you leave the internal system... but a quad bond would make sense...

You're right about heat, but it should suffice to have a fan in front of the drives to keep the airflow good enough to keep the temps low. I encountered that the only problem is the space between disk4 and disk5 where no fan can create an airflow, so you got heatspaces inbetweem them.

Greetings
David

I've never had an issue in a 4 disk raid system. I can always use a 5 disk raid system when I switch to hardware raid. Do you have a link where it show about a 4 disk raid system. As of now my drives are very cool anywhere between 22 and 28 degrees. So no extra cooling is needed. As long as I know where all the drives are it won't matter if I switch to 5 disk raid system. Been using raid for a long time with 4 disks with no problems.

Oh and I forgot to mention the thrird factor the kills hard disks. So for a recap we have.

1. #1 killer is heat
2. #2 killer dead air or no air flow
3. #3 killer is shock

Yup you get lots of DOA devices because it got damaged in transportation or dropping a drive.

As for my 270 megs that's write speeds from one raid array to another. I tested this with Linux mc. My old hardware raid setup would max out at 250 megs again tested with mc. My new hardware raid should get between 500 to 600 megs that's because it has a 1 gig cache of ram ddr3. As for network speed I can always do some network bonding. I have a layer 2 managed switch and it supports bonding so thats not an issue. I'll keep my raid 4 disk system it just makes it so much easier because 4 disks works with my16 bay SAS expanders as well 5 disk the math does not work. Each row had a raid 5 setup. It just works.

Sorry, the Link is german only: https://de.wikipedia.org/wiki/…_Parit.C3.A4tsinformation

Zitat

Einfluss auf die Write-Performance

Im Unterschied zur Read-Performance ist das Ermitteln der Write-Performance bei RAID 5 deutlich komplizierter und hängt sowohl von der zu schreibenden Datenmenge, als auch von der Anzahl der Platten ab.[2] Ausgehend von Festplatten mit weniger als 2TB Plattenplatz, ist die atomare Blockgröße (auch Sektorgröße genannt) der Platten häufig 512 Byte (siehe Festplatte: Speichern und Lesen von Daten). Geht man weiter von einem RAID-5-Verbund mit 5 Platten (4/5 Daten und 1/5 Parität) aus, so ergibt sich folgendes Szenario: Will eine Anwendung 2048 Byte schreiben, wird in diesem günstigen Fall auf alle 5 Platten genau je ein Block zu 512 Byte geschrieben, wobei einer dieser Blöcke keine Nutzdaten enthält. Im Vergleich zu RAID 0 mit 5 Platten ergibt sich daraus eine Effizienz von 80 % (bei RAID 5 mit 3 Platten wären es 66 %). Möchte eine Anwendung nur einen Block von 512 Byte schreiben, so ergibt sich ein ungünstigerer Fall, es müssen zuerst der abzuändernde Block und der Paritätsblock eingelesen werden, danach wird der neue Paritätsblock berechnet und erst dann können beide 512-Byte-Blöcke geschrieben werden. Das bedeutet einen Aufwand von 2 Lesezugriffen und 2 Schreibzugriffen, um einen Block zu speichern. Geht man vereinfacht davon aus, dass Lesen und Schreiben gleich lange dauern, so beträgt die Effizienz in diesem ungünstigsten Fall, dem sogenannten RAID 5 write Penalty, noch 25 %. In der Praxis wird dieser Worst-Case-Fall bei einem RAID 5 mit 5 Platten aber kaum eintreten, denn Dateisysteme haben häufig Blockgrößen von 2 kB, 4 kB und mehr und zeigen daher praktisch ausschließlich das Well-Case-Schreibverhalten. Gleiches gilt analog für RAID 5 mit 3 Platten. Unterschiedlich verhält sich hingegen etwa ein RAID-5-System mit 4 Platten (3/4 Daten und 1/4 Parität), soll hier ein Block von 2048 Byte geschrieben werden, sind zwei Schreibvorgänge notwendig, es werden dann einmal 1536 Byte mit Well-Case-Performance geschrieben und noch einmal 512 Byte mit Worst-Case-Verhalten. Diesem Worst-Case-Verhalten wirken zwar Cache-Strategien entgegen, aber dennoch ergibt sich hieraus, dass bei RAID 5 möglichst ein Verhältnis von zwei, vier oder auch acht Platten für Nutzdaten plus einer Platte für Paritätsdaten eingehalten werden sollte. Daher haben RAID-5-Systeme mit 3, 5 oder 9 Platten ein besonders günstiges Performanceverhalten.

Also read my NAS thread, you can see my benchmarks there, I get over 500mb/s with an old SATAII hardware controller from 3ware/LSI.

Greetings
David