Best data storage strategy for my home NAS

    • Offizieller Beitrag

    What about partition configuration of the disks itself

    OMV uses the whole drive to create the raid.

    Is it ok to create RAID 6 array with btrfs filesystem on top f it

    Yes it's possible as there is the option to format the array with btrfs, the question is why

    that can meet my requirements, that is:

    I did answer these in post 2

    Is there any simple way to support file history using ext4 fs

    ?( what do you mean by simple,as I said according to the MS docs you can use network storage for this, so you create a share with the appropriate permissions then add that share to SMB and it's accessible across the network.

    How stable is zfs these days on OMV

    It's usable, it's a plugin but not sure if it's maintained and some of the config if there is a problem is command line

  • Is there any simple way to support file history using ext4 fs?

    What do you mean exactly with "file history"? Do you mean the "restore previous version" feature of Windows.


    To be able to use this feature with Windows over a NAS share you need a file system on the NAS which supports snapshots. With ZFS this is definitely possible. I should know this, because I have configured this on my NAS.


    P.S: And no, it´s not that simple to configure ;)

    OMV 3.0.100 (Gray style)

    ASRock Rack C2550D4I C0-stepping - 16GB ECC - 6x WD RED 3TB (ZFS 2x3 Striped RaidZ1) - Fractal Design Node 304 -

    3x WD80EMAZ Snapraid / MergerFS-pool via eSATA - 4-Bay ICYCube MB561U3S-4S with fan-mod

  • This it also allows you to use a network location, rather than 'mapping' a share to windows.

    That´s interesting! I didn´t know this feature. Is there a possibility to select which files or folders of the Windows system are backed up to the backup share location?

    OMV 3.0.100 (Gray style)

    ASRock Rack C2550D4I C0-stepping - 16GB ECC - 6x WD RED 3TB (ZFS 2x3 Striped RaidZ1) - Fractal Design Node 304 -

    3x WD80EMAZ Snapraid / MergerFS-pool via eSATA - 4-Bay ICYCube MB561U3S-4S with fan-mod

    Einmal editiert, zuletzt von cabrio_leo ()

    • Offizieller Beitrag

    That´s interesting! I didn´t know this feature.

    TBH neither did I, but I believe you can select what is backed up, as it uses your 'locations' i.e. Documents, Photos, Videos, I wouldn't this as I have 365 for the family. But I also use UrBackup to backup an image of each machine, which I know I can restore.

  • Yes it's possible as there is the option to format the array with btrfs, the question is why

    Correct, and in the first answer we read: "In 2016, Btrfs RAID-6 should not be used." I've already seen this post before, that is why I am uncertain of whether it is ok at 2020 to perform such thing. I think I've already explained why I would like to follow this way, because I would like to have a redundant array. Since RAID 5 these days at current disk volumes is dangerous due to high possibility of a second disk failure, I decided to utilize RAID 6 which has 2 parity disks. Obviously JBOD does not protect against a disk failure which is most common problem in storage devices. If one disk fails, I'll loose all the data from the array in the case of JBOD. I know my solution with RAID 6 might not be perfect since I'm not a skilled professional but that's basically why I'm here is to get an advice on how others organize their storage space and what are other options I might have considered during partitioning. And yes, I remember that RAID is not a backup ;)


    what do you mean by simple,as I said according to the MS docs you can use network storage for this, so you create a share with the appropriate permissions then add that share to SMB and it's accessible across the network.


    It is not that simple as cabrio_leo noted:


    To be able to use this feature with Windows over a NAS share you need a file system on the NAS which supports snapshots. With ZFS this is definitely possible. I should know this, because I have configured this on my NAS.

    I'm talking about Previous version option in Windows, which allows you to restore a previous version of any file where this option is enabled. Especially it is useful for documents which can be modified a lot during short period of time. And yes, it requires a file system which supports snapshots such as btrfs or ZFS. I know that ZFS came from the FreeBSD world and is not well supported by Linux kernel, that is why btrfs was my choice. I'll be happy if you can suggest alternative file systems or their configurations which allows snapshots and well maintained in OMV.


    This it also allows you to use a network location, rather than 'mapping' a share to windows.

    At the moment I don't need to backup files from my computers to NAS. My NAS, basically, is my storage and I'm working directly with the files located on it. Only sometimes I prefer first to copy a file to my PC rather than work with it remotely. But, that is rare, and after I finished my task, I'm copying it back to NAS.

    • Offizieller Beitrag

    I think I've already explained why I would like to follow this way, because I would like to have a redundant array. Since RAID 5 these days at current disk volumes is dangerous due to high possibility of a second disk failure, I decided to utilize RAID 6 which has 2 parity disks. Obviously JBOD does not protect against a disk failure which is most common problem in storage devices. If one disk fails, I'll loose all the data from the array in the case of JBOD.

    I think one of us is missing something somewhere, have a read of this as to snapshots then ZFS is your only choice, your reference to a btrfs raid is still the same today.


    A raid setup is about availability, there is no redundancy and I've certainly never said anything about JBOD, if a controller supports it then then smart is effective and usable, your card doesn't support JBOD, just passthrough. TBH if cabrio_leo can assist you with ZFS then fine, but it's quite different to Freenas and Xigmanas.


    I shall unwatch this thread as I don't thing I can add anything else, good luck with whatever you do.

  • TBH if cabrio_leo can assist you with ZFS then fine, but it's quite different to Freenas and Xigmanas.

    sokol I agree with geaves. If you want to use ZFS in OMV a lot of things must be done by CLI. E.g. there is no mechanism in OMV to create ZFS snapshots automatically on a regular basis. Third party tools must be used for that. But this is a must to support Windows previous versions. Also it is necessary to modify the Samba configuration that the ZFS snapshots are announced as previous versions to the Windows PC. For me this was a lot of trial and error.

    OMV 3.0.100 (Gray style)

    ASRock Rack C2550D4I C0-stepping - 16GB ECC - 6x WD RED 3TB (ZFS 2x3 Striped RaidZ1) - Fractal Design Node 304 -

    3x WD80EMAZ Snapraid / MergerFS-pool via eSATA - 4-Bay ICYCube MB561U3S-4S with fan-mod

    • Offizieller Beitrag

    **Sorry - this post was delayed a bit **

    ZFS has been stable for decades, even on Linux. It's the grand daddy of COW file systems. But understanding the effects and limitations of adding drives to an ZFS array, is another topic. If it was me, I'd use ZFS and go with a RAID 10 (like) configuration with VDEV's made up of zmirrors. When adding two drives at a time, an array can be expanded with no limitations and with high safety, but the cost is 50% of disk realestate. (Typical of RAID 1 and 10.)

    With ZFS, you can get SNAPSHOTS with file versioning good up to one year (or more if configured for it using something like zfs-auto-snapshot).
    Past file versions, even full filesystems states, within snapshots, can be made visible to windows clients with the following command line.
    ________________________________________________


    zfs set snapdir=visible <pool>/<dataset>

    To return to default:

    zfs set snapdir=hidden <pool>/<dataset>


    **For those who may happen onto this post:**
    The above is for file / folder restoration purposes and should be changed back to =hidden when done. It's a bad idea to leave snapshot dir's visible to windows clients and backup servers, long term. Backup app's and command lines that backup your server's files, that are not snapshot aware (most of them), would be backing up several copies of the same file or folder.

    ________________________________________________________________________________

    There's BTRFS to consider but it's still (still) not ready for a RAID6 configuraton. (It's still listed as unstable. BTRFS Status)

    ________________________________________________________________________________

    While it can be done, I don't think there's any real benefit to setting up mdadm RAID6, the formatting it with BTRFS. You'd get error detection and a few minor tools but that would be it. The real benefits wouldn't be available.
    ________________________________________________________________________________

    You might consider SNAPRAID. That can be configured for RAID6 like protection, allowing for the loss of 2 disks. Disk aggregation can be done with Mergerfs which provides a common mount point for multiple disks.

  • I'm also sorry for a short absence.


    My friends, I do believe there is a small misunderstanding. I think I was not clear enough when was asking for an advice especially with the filesystem that supports snapshots. I already have one which is btrfs. I found and modified a very neat script which handles the snapshoting of my partitions. Samba shares configured in such a way that file history is accessible for Windows machines [ crashtest thanks for a clue with hidden snapdir, I believe there should exist something similar for btrfs as well, I'll check it later on]. Actually I have a working NAS which is up and running with plenty of services as dockers and all my files stored in its disks.


    I know that it is a wrong approach to start from the "end" in designing the storage place. Unfortunately I run into information about hardware issues with RAID controllers only very recently when everything was almost finished. Since the stored data in my NAS is important to me, I decided to take a step back and dive deeply into the matter of this issue. During my research I realized that my approach with btrfs and RAID (even its soft version using mdadm) has also a lot of issues with stability and safety.

    Yesterday I picked up an IBM M1015 RAID controller from a local secondhand market for around 26US dollars. It took a lot of time to reflash it in IT mode only due to strange behavior of my motherboard. Mainly I was fighting with running an EFI shell and performing commands in it. Finally I succeeded and now I've got a working HBA/IT card! So there will be no interference of two controllers, i.e. one on the RAID card and another one from the side of the operating system. Minus one problem :)

    Since most definitely I'll be formatting my array due to peculiarities of current RAID controller, I'll need to figure out which approach to follow next. Obviously it should be a more reliable one, regardless of necessity to work in the command line (some minor shell scripting is acceptable especially if there are existing templates). I did manage to run dockers and keep their configs on the SSD with the OS, so this operation will not require a lot of routine work with restoring their functionality. Also I have backuped my system and all files.


    Summarizing above said, I am looking for a solution reliable against a most common problem such as drive failure, which is capable of snapshoting and is relatively easy to maintain. I strongly believe that this is something very common in the world of home NAS and there are plenty regular users which have similar requests to their storage place and already adopted good strategies.

    • Offizieller Beitrag

    I strongly believe that this is something very common in the world of home NAS and there are plenty regular users which have similar requests to their storage place and already adopted good strategies.

    You might be surprised. Most users seem interested in the maximum storage space possible, and are begrudging of the drives / space required for a measure of safety. The majority think that some form of RAID protects them from data loss (not true) and completely forego backup. Bitrot protection and correction is understood only by a very few.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!