Is RAID5 really that bad?

  • Loaded subject, I grant you that.


    I have used RAID5 for a while but have started seeing more and more online about how RAID5 is dangerous and should be avoided due to write delay and parity errors. What are your thoughts?


    I would love to do a poll on here for what level of raid people use. I know it is often application specific but it would be interesting to see.

    • Offizieller Beitrag

    Been using raid 5 for a long time without any issues. I have raid 0,1,5, and 10 arrays across various machines. So, your poll is tough to answer :)

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hardware Raidcontroller 8-)


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Yes, yes it is. I have looked after several Raid 5 based business class NAS units for a couple of years. In that time I have have total data loss twice using RAID5 (luckily I had backups).


    Based on my experience the following use case is horrible for RAID5 and will result in data loss:


    1) Large disks (>1TB) these drives take a long time to rebuild in the event of a failure. Because of the huge amount of data, rebuilding the array depends on being able to read the remaining drives (see number 2)
    2) Large number of disks - the more disks in your array the higher the likelihood that a drive will have errors that prevent rebuild from accessing some of the information on the drive
    3) Consumer Class Disks drives fail much more often than the more expensive enterprise class drives (not really a RAID 5 specific issue but which drives does your array use?)
    4) Archive Data - data such as pictures, video, etc. that is not routinely overwritten and only periodically read. This type of usage does not typically allow the controller or the drive to sense that some sectors are becoming unreliable. The drive sectors can be unreadable and until you attempt to read the data for a rebuild you have no idea. Video streaming is often fairly forgiving and will probably just continue when certain data cannot be read.


    It has been my experience with 8TB - 10 drive arrays using SATA drives you will lose data once a year. If you have a choice pay extra for RAID6. If not ensure that you have a copy of the data on another system.


    Any number of sources indicate the same for example: http://www.hardwaresecrets.com…-Over-RAID0-and-RAID5/314


    As for the Hardware Raid Controllers that someone mentioned, I think I would prefer software RAID like Linux can use. The controllers have smarts built in to sense drive failures and will helpfully take the array offline when that second drive fails. The vendor did not seem to have any help for recovering data but I suspect that there would be a way to convince software raid to keep the array online to recover more data.


    regards


    Tim

  • Zitat von "timlegge"

    Yes, yes it is. I have looked after several Raid 5 based business class NAS units for a couple of years. In that time I have have total data loss twice using RAID5 (luckily I had backups).


    Based on my experience the following use case is horrible for RAID5 and will result in data loss:


    1) Large disks (>1TB) these drives take a long time to rebuild in the event of a failure. Because of the huge amount of data, rebuilding the array depends on being able to read the remaining drives (see number 2)


    Sure, if two drives fail, you are screwed. Because of that Business Class uses RAID 6 or properitary mechanisms. Rebuild? Under 7 hours for a three terrabyte disk . Yes, a hardware RAID operates as fast as the disk underlying it. And you know what? You don't have to spend much money on it. And I am talking about a 5400rpm drive here.


    Zitat von "timlegge"

    2) Large number of disks - the more disks in your array the higher the likelihood that a drive will have errors that prevent rebuild from accessing some of the information on the drive


    Simple comment: Bullshit. If you have the fear or the thinking that this could happen, buy disks from different Vendors to get different charges of your drives. Or if you want to use a large amount of disks, either split them for 2 arrays, or go with RAID 6.


    Zitat von "timlegge"

    3) Consumer Class Disks drives fail much more often than the more expensive enterprise class drives (not really a RAID 5 specific issue but which drives does your array use?)


    You talk about Business Class in the beginning. So, why do you use Business Class NAS Equipment, when you fill it with crappy Desktop Class Hard Disks? By the way, Desktop Class Hard Drives are only certified to run 8 hours a day, 5 days a week. But let me hint you, that there are now "NAS Class" Drives out there. The WD Red are designed for Home NAS usage. They are certified for running 24/7 and have a extended three years warranty. Also they got some of the features of there Business Class brothers.


    Zitat von "timlegge"


    4) Archive Data - data such as pictures, video, etc. that is not routinely overwritten and only periodically read. This type of usage does not typically allow the controller or the drive to sense that some sectors are becoming unreliable. The drive sectors can be unreadable and until you attempt to read the data for a rebuild you have no idea. Video streaming is often fairly forgiving and will probably just continue when certain data cannot be read.


    And thats why Hardware Raidcontrollers do a Verify every week on standard Settings. I don't know much about Software Raid, but I guess it does too. You're right on the Part that Video, and therefore all kinds of Media, are forgiving if certain bits are missing.



    Zitat von "timlegge"

    It has been my experience with 8TB - 10 drive arrays using SATA drives you will lose data once a year.


    Can you prove that with any MTBF? Percentage of drive failures within the warranty of Drive Manufactures?


    Zitat von "timlegge"

    If you have a choice pay extra for RAID6. If not ensure that you have a copy of the data on another system.


    Allways ensure you have another Copy. Don't get false trust in any RAID System.


    Zitat von "timlegge"

    Any number of sources indicate the same for example: http://www.hardwaresecrets.com…-Over-RAID0-and-RAID5/314


    You are refering to an Article that has the headline, why a RAID with two parity Drives is better than a RAID, that, for the first, doesn't even is a qualified RAID Type and for the second, only has one parity Drive? I can tell you in two sentences.
    RAID 0 is no RAID, a drive goes and so goes your data.
    I don't think you need to tell anyone, that two parity Disks is better than one.


    Zitat von "timlegge"

    As for the Hardware Raid Controllers that someone mentioned, I think I would prefer software RAID like Linux can use. The controllers have smarts built in to sense drive failures and will helpfully take the array offline when that second drive fails.


    They don't. This could lead to a bunch of problems, so no, they don't. But if enabled, you would get imediate Information about the Drive failure.


    Zitat von "timlegge"

    The vendor did not seem to have any help for recovering data but I suspect that there would be a way to convince software raid to keep the array online to recover more data.


    Even if one drive fails, a hardware RAID 5 still operates. So don't bother, you could still recover your files.


    Let me tell you something, in the past 10 Years i only had Drive problems twice. The first time, I got some Bad Sectors at the moment i was replacing a Disk it with a new one. If I remember correctly, a handfull Video Files (replaceable) got damaged and were unable to recover. The Second was after a Drive failed in my RAID5 Array and i screwed it, because i touched the cables of my Hard Disks and got a second Drive to jump out of the Array. You think i might were screwed at the moment? No, because the Vendor of my 3ware Controller, which is LSI now, helped me, to force that Drive back into the Array. And you know what, I recovered all of my 4.5 terrabyte of data.


    Zitat von "timlegge"

    regards


    Tim

    [/quote][/quote]


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • If you trust your data to be stored in only one place, raid or otherwise, you really do not have your data. Ideally you need your data in three places - viewtopic.php?f=3&t=2050. This would apply for anyone, business and so forth.


    So backup, backup and backup once again.

  • My raid5 = 4 x 2Tb Seagate.
    If the raid a little tweak you can not be afraid of the big time synchronization.
    When I began to create a raid5 with the default settings, then presumably resyncing time was about 70,000 minutes.


    Then I
    # echo 8192 > /sys/block/md0/md/stripe_cache_size
    changes occur immediately and time resyncing decreased to 400 minutes:
    see attachment screen-000-3.png


    after the array is created I got the following results:
    # time dd if=/dev/zero of=/media/07719c31-9381-443f-aeb2-6d0d2e0232e2/files bs=1M count=5000
    Speed was such a 146Mb/s
    # time dd if=/media/07719c31-9381-443f-aeb2-6d0d2e0232e2/files of=/dev/null bs=1M count=5000
    Speed was such a 281Mb/s


    it seemed to me a little and I did so:
    # echo 16384 > /sys/block/md0/md/stripe_cache_size
    and the results were such:
    # time dd if=/dev/zero of=/media/07719c31-9381-443f-aeb2-6d0d2e0232e2/files bs=1M count=5000
    Speed was such a 275Mb/s
    # time dd if=/media/07719c31-9381-443f-aeb2-6d0d2e0232e2/files of=/dev/null bs=1M count=5000
    Speed was such a 283Mb/s


    I decided to stop here and added the line
    echo 16384> / sys/block/md0/md/stripe_cache_sitse
    in /etc/rc.local


    The software supports raid wonderful piece: bitmap. There are marked changed blocks to disk, and if you somehow fell off one drive of the array, and then you add it back - a complete overhaul of the array is not needed. Very useful.
    # mdadm -G /dev/md0 -b /var/md0_intent


    I enjoyed the articles here:
    _http://habrahabr.ru/post/111036/
    _http://romanrm.ru/mdadm-raid
    You can read them via translate.google.com


    P.S. I am sorry for my bad english :)

  • I just wanted to mention my experience without raid:
    - for about 15 Years now i am working with PCs and since i had my first ISDN Flat here in germany i was in need of much disk space ...
    - within these 15 Years i lost my mp3s once and some private pictures once. But only! because of my own stupidity! because i never did a backup and accidentially deleted the data / installed windows/debian/ubuntu on the wrong drive (overwriting the data mostly)
    - i never used any kind of RAID until this year and i am glad to reduce the potential of a disc failure. Now one disc can completely fail before i would lose any data.
    - now i own a second hp microserver which will become the historical backup system for the first. Then two discs may fail and i could additionally accidentially wipe one complete system.


    As conclusion:
    RAID 5 is more secure than no RAID
    There are more secure RAIDs where two HDDs may fail at one time (at the cost of space, cash ...)
    In 15 years not even one HDD failed in my home


    With this kind of backup/security i think i am quite safe. In every case a lot safer than in the 90s :D

  • I have a HP microserver with 4x2TB drives, configured in RAID5. I backup really important stuff, like personal documents and photos offsite. But It's not been practical to backup the whole thing, at least not regularly. Besides, a lot of the content are ripped movies which I can re-rip if necessary (I suppose in a sense you could argue the physical discs are already a type of backup).


    A few months ago I had a drive fail. Popped it out, put in a new one and let the RAID array rebuild. Utterly brilliant. I saved a LOT of time, and potentially saved a lot of data too.


    That said, I'm starting to re-think the whole thing. The fact that all the disks in the array run 24x7 bothers me... both in terms of power efficiency and increased wear-and-tear. Plus, the fact that a second drive failure would result in losing everything.


    I'm starting to look at switching over to FlexRAID or SnapRAID as potential alternatives. They both deal with some of the inherent issues with RAID5 and offer greater flexibility. There are some downsides, particularly with SnapRAID. But in the right situation the positives far outweigh the negatives.

  • Tyberious Funk:


    Migrate to raid 6...


    6watt per disk is not much....


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • We all have to make sacrifices for our needs.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • For me I prefer raid 5 to raid 6 and I'll explain why. First raid 5 is faster than raid 6 due to only one parity calculation vs 2 for raid 6. Also raid 5 gives more storage space to raid 6 due to only 1 drive being lost vs 2 to parity. However raid 5 in a large number of disks is a bad idea because the more disks you have in a single array the more of a chance that 2 drives will die at the same time. The way to solve this is to put the raid 5 array in small groups. In my current setup I have 3 software arrays of 4 disks each in raid 5. All software. I have a Norco 4220 and each row is a raid 5 array of 4 disks. I have 12 X 3TB WD RED drives in 3 raid 5 arrays. I also have my old hardware raid 5 which is 8 X Seagate 500G enterprise drives in 2 separate array of 4 disks per row. I have everything managed with LVM via the OMV gui. Its flawless both are the same speed go figure. The software raid 5 I get 270 megs peak and the hardware raid I get 250 megs peek.


    This way if I loose and array it's limited to only one of my 5 arrays where if I had put all my disks in 1 array and 2 drives would die at the same time I would loose 100% of data. The LVM is just a container to manage all the separate arrays accept for the hardware raid where the array is managed in the card firmware. OMV sees the arrays as 2 large separate disks. The software array is managed via OMV gui. So its 3 software raid 5 arrays of 12 disks total and 2 hardware arrays of 8 disks total.


    This whole setup will change when I go all VM Xenserver. One central machine to have all my hardware and if I need more space I expand via sas expanders. This new setup will save me on my monthly electric bill.

  • Cpoc, as much as I like your thoughts, you stumble over your own thoughts. You say you don't like RAID6 because it needs double the cpu power. Because of that you use RAID5. Because RAID5 gets more risk with more discs you split your disks into multiple (three) arrays. Lets do the math quickly: You need 3 times the cpu power instead of a single RAID5 and 1,5 times the cpu power than a full RAID6.


    Also your suggestion that if you put all disks into one array you would loose all data when two disks are failing is not correct I think because you would use RAID6 instead of five on such a big array!


    Also as a reply to the other thread, what do you mean with "only good hardware raid controllers with 1gb of ram"? IIRC my controller only has 256MB ram and it runs like hell, as you can see in my benchmarks.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • My current setup is going to change in very near future. What I meant is allot of old raid cards don't support more advanced features like smart,ncq,trim for ssd and so on. The new ones do. My old adaptec raid card is solid like hell but does not support smart, and has a limitation of only 2 terabyte per array so thus why its still married to 4 x 500G drives which is under 2 terabytes. The raid is flawless but starting to show its age. I don't know which model raid card you have but in general the old raid cards are not recommended. My new raid card will be the LSI 9266-8i with 1 gig ram. The reason I'm going with that raid card is just because its built into the motherboard so hence free hardware raid. Add that to my HP 36port sas expander and I have lots of room for expansion all hardware raid.


    You don't have to show me any benchmarks I believe you. You cannot compare the speed of software raid to a high end hardware raid. If you are planning to buy hardware raid just make its a good one that will last you many years to come.

  • I "just" recently upgraded my controller, but its the same series, just the bigger model:


    http://forums.openmediavault.org/viewtopic.php?f=8&t=1960


    (lsi) 3ware 9650se-16ml


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Yeah except that you have to access the smart information manually (you can not access the drives directly through the omv gui and the gui doesn't support to access the virtual drive) the controller (including the old one) never let me down, even when I had two drive failures in a raid5 (:O) I was able to recover my files.


    Greetings
    David


    PS: Maybe an idea for a new plugin...

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!