Best data storage strategy for my home NAS

  • Hi to all,


    Finally I built my first NAS based on OMV 5 using most of components left from previous computer builds. Before I used a QNAP branded NAS to store my data on. At some point I realized it was not enough for my purposes and I decided to build a network storage on my own. For this purpose I picked up an AM1 motherboard with AMD 5350 APU on board. I expanded my existing storage capacity to 5x2Tb drives for the most important part, i.e. documents, photos, storage for nextcloud, programs; 10Tb drive for torrents and 256Gb SSD for the operating system. The motherboard contains only two SATA connectors, so I attached a 3ware 9750 RAID controller to it in order to expand my storage capability. Basically only 2Tb drives are connected to the RAID controller since it is too old to read drives larger than 2Tb. So 10Tb and SSD are attached to the motherboard directly.


    Here is the main part of my story. Going over information posted in this forum and other places in the internet, I understood a few things. First of all, a hardware RAID is more difficult to rebuild in case of a hardware failure with the controller. Much easier for manipulation is software raid. A good choice for future expandability of my array is LVM. So I ended with mdadm and LVM on top of it. Next I formatted my drives to EXT4 as advised here as the easiest and most reliable file system especially for newbies. However I wasn't satisfied with it, since this fs doesn't support snapshots and as a result file history in Windows. I wanted to have this option available especially for documents and photos. After that I reformatted my array into btrfs. So at the moment my config is as follows: LVM on top of mdamd with btrfs as a main filesystem (except partition for nextcloud where I left ext4).


    Reading in the internet about possible problems with such configuration I realized that this is more likely an overkill (it seems btrfs can easily be expanded without need to have LVM structure) which can bring to many issues once I need to rebuild an array or restore data. So, I need your help guys, what is the best storage strategy enabling the following features:

    1. Redundancy with 2 parity drives (similar to RAID 6). In case of failure of one of the disks (or RAID controller) the array should be restored easily.
    2. Expandability. Necessary partitions should be easily expanded in case of adding another 2Tb drive to the system.
    3. Support for File history in Windows.
    4. Reasonably simple array maintenance.

    Looking forward for practical advises from any of you.

  • First of all, a hardware RAID is more difficult to rebuild in case of a hardware failure with the controller. Much easier for manipulation is software raid

    That's partially true as the Raid controller is only accessible during post where the Raid is configured, this is fine on MS Servers as there is usually additional software to monitor the Raid.


    LVM as you have discovered this is not necessary, as to btrfs, :/ another user asked the question as he was currently using either a Qnap or Synology, turns out one them uses mdadm with LVM then formats to btrfs rather than using a btrfs raid set up.


    My question is, does that card support IT mode (HBA) doing a search doesn't give definitive answers.


    So to your questions;


    1. If you are planning on using more than 4 drives then raid 6 would be the sensible option, but if more than 2 drives fail the raid is toast. If the controller failed connecting the drives to another raid card should work as the metadata is on the drives.


    2. You can use the WebUI, connect a drive, wipe it first, then in raid management select the raid and use recover on the menu to add the new drive.


    3. Interesting, as the whole house uses Windows we don't use this feature as I backup using UrBackup, but I assume you could set up an SMB share to do this, at least according to MS docs. One thing I never do on my Windows machines is to map a network location as a drive letter, two recent threads on here highlighted my reasoning.


    4. :D:D software raid (mdadm) is what it says, software, you have to tell it what to do, some can be done in the WebUI but there are cases where a lot of it is done via the command line. The biggest problem users have is power outages that can cause the array to be become 'inactive', another is replacing a drive that's failing mdadm is not hot swap


    With any system a good backup strategy is the key, hence my sig, the number of users who believe that their data is safe should a drive/s fail I've given up counting.

  • geaves did a good job of describing some of your RAID options, but his best advise is "backup".

    So many get caught in settling on a "safe" storage method, that they forget backup. With 100% data backup, setup in a simple manner, the risks associated with using a storage method you're not familiar will drop dramatically.

    Backup doesn't have to be complicated. You could use a big external drive with the main server, or an SBC together with an external drive. (An independent second host is preferred.)

    Give it some thought.

  • geaves thanks for the answer.


    That's partially true as the Raid controller is only accessible during post where the Raid is configured, this is fine on MS Servers as there is usually additional software to monitor the Raid.

    Actually my controller has an option to sent me emails on failures or possible problems. So it can be an alternative to have a fully functional WebGui monitor software. Yes, it is not the best but, at least I am always updated.

    My question is, does that card support IT mode (HBA) doing a search doesn't give definitive answers.

    I believe it does not. I've read many times about this recommendation in this and many other forums devoted to different NAS software solutions. I configured my RAID controller to passthrough all drives. I assume it actually created some sort of a RAID 0 containing one drive, since I can't find any information on pure HBA mode for this controller. Yet OMV can see all the drives separately in the WebUI, so I was able to configure as I wanted.


    1. If you are planning on using more than 4 drives then raid 6 would be the sensible option, but if more than 2 drives fail the raid is toast. If the controller failed connecting the drives to another raid card should work as the metadata is on the drives.

    Yes, I agree with you. At the beginning I was thinking about RAID 5 until I'd discovered problems with high possibility of another disk failure during array rebuild which will destroy the array completely. I know that it is especially critical for large disks due to hardware limit for number of read/writes, but still I did not want to put in risk all my data.

    What I've learned from the Internet that in case of a hardware RAID in order to restore your data from the array you MUST connect it to the same raid controller model. Which is not the case with the software one. So who is right? :)

    2. You can use the WebUI, connect a drive, wipe it first, then in raid management select the raid and use recover on the menu to add the new drive.

    I'm not sure I will be able to expand existing partitions if I'm not using appropriate software configuration such as LVM, for example.

    3. Interesting, as the whole house uses Windows we don't use this feature as I backup using UrBackup, but I assume you could set up an SMB share to do this, at least according to MS docs. One thing I never do on my Windows machines is to map a network location as a drive letter, two recent threads on here highlighted my reasoning.

    My NAS is my storage, I'm working on it directly. I don't know if it is a good or bad idea, but I wanted to be able to access my data remotely from any computer in my house. That is why I need File history available on it.

    Can you point me to the discussion regarding the drive letters? I hear this for the first time.


    4. :D:D software raid (mdadm) is what it says, software, you have to tell it what to do, some can be done in the WebUI but there are cases where a lot of it is done via the command line. The biggest problem users have is power outages that can cause the array to be become 'inactive', another is replacing a drive that's failing mdadm is not hot swap

    Yes, I'm aware that mdadm is not hot swappable. I'm ok with it.

    With any system a good backup strategy is the key, hence my sig, the number of users who believe that their data is safe should a drive/s fail I've given up counting.

    Completely agree with this! That is why I configured my old QNAP to be a remote backup storage for my current NAS. Once I finish configuration of both of them I will place the old one to my parents house.

  • Actually my controller has an option to sent me emails on failures or possible problems. So it can be an alternative to have a fully functional WebGui monitor software. Yes, it is not the best but, at least I am always updated.

    Interesting, not seen that as an option, mind you I have only ever used Adaptec on MS Servers.

    Yet OMV can see all the drives separately in the WebUI, so I was able to configure as I wanted.

    I doubt it creates a Raid 0 there must be something in the cards Bios that allows passthrough once configured, in essence what you want and what you have tested OMV needs to 'see' drives individually to configure mdadm.

    What I've learned from the Internet that in case of a hardware RAID in order to restore your data from the array you MUST connect it to the same raid controller model. Which is not the case with the software one. So who is right?

    That's true if you were to use the card as a true hardware raid, by allowing passthrough the drives are individual, so if you had to replace the card it would have to be something that supported HBA, or a card that could be flashed to IT mode.

    I'm not sure I will be able to expand existing partitions if I'm not using appropriate software configuration such as LVM

    You will not need to use LVM, the Raid Management UI or the command line is all you need, no need to over complicate it.

    Can you point me to the discussion regarding the drive letters

    One is still open under the subforum SMB/CIFS with that one it's down to authentication, somewhere, the SMB set up is the same as mine except I do not map network shares to a drive letter on any of my windows machines, I don't get any problems.

    The other was a few weeks ago where user posted that his OMV had been infected with a virus, it wasn't a virus it was ransomeware, whilst there was no outcome, in fact never heard any more. The ransomeware either came from a windows machine through an email or software then populated the 'drives' on the machine which would spread to OMV. Or there was something on OMV that was exposed, the thread went dead when it was suggested he just restore from backup.

    Personally for that Windows Files History I would attempt a direct connection to an SMB share rather drive mapping.

    Yes, I'm aware that mdadm is not hot swappable. I'm ok with it.

    That's fine but some new users to software raid believe it works the same as a Qnap or Synology.

    Completely agree with this! That is why I configured my old QNAP to be a remote backup storage for my current NAS. Once I finish configuration of both of them I will place the old one to my parents house

    8| a user using raid agrees that he should have a backup, that's gotta be a first :D:D

  • That's true if you were to use the card as a true hardware raid, by allowing passthrough the drives are individual, so if you had to replace the card it would have to be something that supported HBA, or a card that could be flashed to IT mode.

    Did not get it completely. So, in case of a hardware RAID when the controller is taking care of array and the OS recognizes it as one disk, it would be a problem to restore data due to necessity to use the same card? Or even in my case when all hard drives are accessible in OMV despite my card does not support HBA I'll have problems once I need to replace the RAID card?


    except I do not map network shares to a drive letter on any of my windows machines, I don't get any problems.

    So how do you connect to the shares on OMV from your Windows machine?

  • So, in case of a hardware RAID when the controller is taking care of array and the OS recognizes it as one disk, it would be a problem to restore data due to necessity to use the same card?

    Yes, that's my understanding.

    Or even in my case when all hard drives are accessible in OMV despite my card does not support HBA I'll have problems once I need to replace the RAID card?

    The passthrough would suggest it's an implementation of HBA, so the raid information from using mdadm will be stored on the drive/s.

  • I suggest looking into whether that 3ware 9750 RAID controller can be updated via new firmware to allow drives larger than 2TB to be used and also be capable of running in IT mode so you can do software RAID if you really want to use RAID. If it can't be updated I suggest abandoning it. A modern replacement card can be had for not much money these days, and the ones that are IT mode HBA only cost less. Even more money can be saved in the used market.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

  • gderf


    The firmware has the latest version. At the moment I'm ok with the card limitation, since my main drives are 2TB. For future upgrade I'll consider a more modern card with HBA/IT support.


    geaves

    I doubt it creates a Raid 0 there must be something in the cards Bios that allows passthrough once configured, in essence what you want and what you have tested OMV needs to 'see' drives individually to configure mdadm.

    This is what I see in the WebGui.



    I have a feeling that each disk has some extra layer from the controller, so if I unplug it and connect to another PC, I would not be able to read information directly. I'll need to check what option I chose when I was configuring the disks. SMART info of these disks is only available through the command line

  • This is what I see in the WebGui.

    That's interesting, I have been doing some research about the card and found this I also found more information from broadcom it appears the one downside to the 9750 is does not support JBOD but does support single disk. That might account for the output you are seeing in storage disks, further broadcom documentation states that the 3ware is designed for use with drives beyond 2TB.

    I have a feeling that each disk has some extra layer from the controller, so if I unplug it and connect to another PC, I would not be able to read information directly. I'll need to check what option I chose when I was configuring the disks

    That makes sense, configure and format one drive in OMV, shutdown remove the drive and test in another machine.

    SMART info of these disks is only available through the command line

    That's a bummer!

  • That's interesting, I have been doing some research about the card and found this I also found more information from broadcom it appears the one downside to the 9750 is does not support JBOD but does support single disk.

    Yes, I think I activated Single disk option for all disks. So I assume, that in this case any hardware problem with the raid controller will automatically make all data unreadable unless I use the same card, correct?


    What are the drawbacks of letting the card to handle the RAID array, apart from cases when it fails? Is their a difference in reliability between file systems, such as ext4 or btrfs, and the data recovering in case of a failure?

  • Yes, I think I activated Single disk option for all disks. So I assume, that in this case any hardware problem with the raid controller will automatically make all data unreadable unless I use the same card, correct

    That's something you'll have to test, at least that's how I would approach it, wipe one drive in OMV, format it to ext4, create a shared folder, shutdown, remove the drive and connect to another Linux machine and see if it can be mounted and read.

    What are the drawbacks of letting the card to handle the RAID array, apart from cases when it fails? Is their a difference in reliability between file systems, such as ext4 or btrfs, and the data recovering in case of a failure?

    Failure of the card or a drive is the only drawback, but a drive failure as you pointed out earlier you can get email alerts from the card, even that you should be able to test, set up a raid 5 or a mirror, pull a drive i.e. remove the power and see if that works.


    As you have not set this up yet you have a number of options you could test to see what suits you best and get an idea on how the card behaves, this is how I would approach it. You already have something in play you just want to expand your storage capacity and installing OMV doesn't take long.


    As I've no experience of this type of card I'm looking at it from a testing point of view, with the thing to remember 'can I access the data if A or B happens'. The one thing you do know if the card fails and you use the card as a raid controller then you'll have to get another like for like card to get access to the data.

  • Yes, I think I activated Single disk option for all disks.

    Noting that this is only my opinion:
    You really don't want a card putting it's finger print on your drives or to use a card that doesn't do fully transparent SMART pass through. In either case, there's potential that that attached drives wouldn't be "portable", or moveable to (for example) a standard SATA port on a motherboard. (As geaves has already suggested, testing is the only way to be sure.)

    I had an Adaptec RAID controller that worked in a similar manner, with a single disk mode available to avoid RAID configurations. Also, it only partially passed SMART data to the host. For that reason alone, I dumped it. Part of what will keep you out of trouble, when using OMV, are the optional e-mail notifications of hard drive events - specifically when SMART counts (errors) increment. Anything that interferes with the interchange of SMART data, between a drive and the host, is not good.

    There are plenty of reasonable priced HBA's on the used market, that can be flashed to full IT mode (fully transparent pass through). Here's a thread of RAID HBA's that can be flashed to IT mode and ref's to a couple flashing processes. Low cost, flashable, HBA's As noted in the thread, I went with the Dell PERC H200.

  • There are plenty of reasonable priced HBA's on the used market, that can be flashed to full IT mode (fully transparent pass through). Here's a thread of RAID HBA's that can be flashed to IT mode and ref's to a couple flashing processes. Low cost, flashable, HBA's As noted in the thread, I went with the Dell PERC H200.

    TBH I think that's the best option the 3ware card could be more trouble than it's worth, prices on ebay UK are about £30 I suppose in the US they're about $20 :)

  • On US ebay Dell PERC H200 cards are sub $50US with quite a few for $30.


    I paid $60 for an new LSI 9200-8E HBA card from Amazon.


    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

  • As noted in the thread, I went with the Dell PERC H200.

    That's funny, but I've also got an eye on this card today :)

    After reading other sources I realize exactly what you said about risks when controller is interfering with the OS. So I decided to find a proper replacement for a reasonable value/price.


    On US ebay Dell PERC H200 cards are sub $50US with quite a few for $30.

    Here it costs only 24$. Is there something wrong with the Chinese versions? I found plenty of them on ebay at similar price tag.

  • I wouldn't accept a month or more for shipping from China to save just a few dollars. If there is a problem with the card when you finally do get it, you can get your money back, but not the lost time.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

    Edited once, last by gderf ().

  • gderf has a point about waiting a month. In the thread, another guy waited a month for a card from China and they canceled the purchase (after a month) so he had to start over. He got one off of Amazon. I got mine from Singapore which came in a bit over 2 weeks but it's also worth noting that I wasn't in a rush.


    Here it costs only 24$

    Don't forget to order brake out cables. As I remember the cables were around $10 US each. (The cable is SFF8087)

    This listing mentions "IT mode", it's in the US, with free shipping, and it includes two cables. Just to be absolutely sure, I think I'd ask the seller if it's already flashed to IT mode.
    If the pictured BIOS screen in the listing is correct, it appears to be flashed to a SAS9211-8i (what you want), with firmware revision XX-IT (done).
    If it's in IT mode already, it might be worth a couple extra bucks to avoid flashing it yourself.


    (Flashing is doable, if following the instructions carefully, but it's not for the feint of heart.)

  • Ok, it is clear now about the controller, which is supposed to be capable to work in IT/HBA mode. What about partition configuration of the disks itself? Does anyone have any other thought or suggestion? Is it ok to create RAID 6 array with btrfs filesystem on top f it, or there are other better options that can meet my requirements, that is:


    1. Redundancy with 2 parity drives (similar to RAID 6). In case of failure of one of the disks (or RAID controller) the array should be restored easily.
    4. Expandability. Necessary partitions should be easily expanded in case of adding another 2Tb drive to the system.
    3. Support for File history in Windows.
    4. Reasonably simple array maintenance.

    Is there any simple way to support file history using ext4 fs? How stable is zfs these days on OMV?

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!