Novice setting up home NAS - after opinions and help if possible!

  • Hi all, I am posting here hoping for a bit of advice, or direction towards guides which may answer my questions but I have missed. I've tried to read all I can and answer my questions myself, but I feel they are probably too specific to be covered and came here for some help!



    What I am trying to achieve: I would like a low power consuming NAS, from which I can serve my files, run a media server, do some light torrenting, and (ideally) backup my important data.



    My old setup: (Very) Old Dell optiplex, running Windows 10, with a JBOD setup of 5 data drives totaling 18TB (1 x 8TB, 1 x 5TB, 1 x 3TB, 2 x 1TB), all NTFS, shared on the network. Also ran plex and deluge from windows on this box. Some of the drives were very old and not in great shape (mainly the smaller three) and my only backup was a recently purchased 8TB USB3 HDD (which I would manually back up to fairly infrequently). Of the 18TB available, I would say that around 15TB was actually used, so I also needed to increase storage shortly.



    So after lots of researching, I couldn't decide what would be the best solution for me. I have really only scratched the surface of linux before, but had some experience with raspberry pi usage (I am currently running a 3b+ with PiHole on it to reduce adverts at home), but once reading that the Pi 4 was pretty good for plex and as a NAS with the USB 3 etc, I decided to make some purchases and go from there, so I bought myself a Pi 4, 4GB model, a powered USB 3 hub, and two new 8TB USB3 HDDs, and started researching.



    My current setup: Raspberry Pi 4, running OMV, with snapraid and unionfs, with my three 8TB USB3 HDDs connected through the powered hub to the Pi. One drive is dedicated to parity, the others are allocated for data. I have managed to get deluge and plex running through docker / portainer (although even when following guides for this I feel like its a bit hit and miss. Deluge seems to be stable now I have the permissions figured out, but I broke Plex again yesterday after configuring the snapraid/unionfs).



    Some questions: 1. Do I need to (or should I, given my usage case) run unionfs as well as snapraid? The way I configured it was to set up the snapraid first (the guides I read seemed to say that it doesn't matter which way round you do this), then add the three drives (two data, one parity) in unionfs to show as one large storage, filling the drive with most free space first. a) was I wrong to add the parity drive in unionfs? b) is it even worth using unionfs in my usage case - it seems to me that equalling the drive usaage would be a good thing, but I am really confused here.
    2. Is my setup sensible? I am currently copying data from my old drives to the new snapraid/unionfs share and once this is completed I hope to sync the snapraid. Once this is done, and I am happy that the data is all there, I was planning on adding some of my old drives to the setup. a) can I easily do this and should I wipe / format the drives? They are all NTFS and I presume that ext4 would be preferrable (the three 8TB drives I am using now are all wiped/formatted ext4 via OMV).



    3. How risky is it playing around with the OMV setup when I have the data on my snapraid array? I figured from the reading I've done that it is fairly safe (which is why I and so many others are choosing it) but are there any big no-nos that I should be aware of? Docker, portainer etc??



    4. Are there any optimisations I could perform? In reading as much as I can for this, there are quite a few topics that I've come across but not had much time to look into properly. For example, alternatives to plex - is emby any good and is it worth changing from plex (which I understand fairly well). Is it worth migrating my pihole setup to this raspberry pi?



    I am sure that there are other questions I meant to ask and more that will pop up in the future, but honestly if you can provide any kind of opinion or help at all that would be massively appreciated.



    Sorry for the huge post, but I figured more information was better than not enough!



    Thanks in advance, and I look forward to learning from you guys.

  • Hi


    I just changed my Nas to OMVtwo weeks ago. Here is what I learned so far:


    • I installed two (empty) Harddisks (to be configured as RAID)
    • The OMV system eas installed on a 128GB USB Stick.
    • Using the web interface the two harddisks were configured as RAID (Mirror), formatted and the first File system was created
    • Thereafter I started copying the first Data (only a few) from the old NAS to the OMV and checked that everything turned out find. I did!

    Then I installed a different version of OMV onto the USB stick to see what happens:
    The good news is:

    • The Raid is still there
    • The file system however shows empty, but if you create a new file system with the same folder name as before, all your copied files appear again.

    I did about 6 new installations witout any data loss.


    Have fun!

  • Thanks for your reply. It's good to know that you have reinstalled the OS without any problems.


    I am guessing from you saying that you have two mirrored drives that you are running a standard RAID (RAID1?).


    Standard RAID does not really fit my use case as far as I can tell. Snapraid provides some form of backup, while allowing me to 'spend' much less HDD space on parity (or mirroring). I can still recover from hardware failures and as an added bonus have backups for things like accidental file deletions (if I am understanding it correctly).


    I guess my most pressing question at the moment is: Do I gain anything by using unionfs as well as snapraid? If I mount a samba share on a snapraid array, won't it present as the whole available storage space, and will it not fill disks evenly?


    What (if anything) will I lose out on by not using BOTH unionfs and snapraid? Is snapraid alone not recommended:?


    Thanks

  • Do I gain anything by using unionfs as well as snapraid?

    Yes, using mergerfs (unionfs) you have a single mount point, but this is where new users get confused (I did), my mergerfs is set as MFS (Most Free Space) that means a file is copied to the drive with the most free space, some use LFS (Least Free Space). LFS will fill the drives sequentially, I think! Most new users have their setting to Existing Path Most Free Space and this is where problems set in, you'll need to search the forum to find some definitive answers.
    There is a caveat to using mergerfs (unionfs) you cannot point docker or docker configs to mergerfs mount point it simply doesn't work!! you either have to point those to a single drive within the mount point or use a separate drive. Likewise if you intend on using UrbackUP like I do, the db gets corrupted using the mount point.
    SnapRaid sits on top of mergerfs and makes things easier to manage, I'm not the best person to explain this as I'm still learning myself. SnapRaid is fine if you are using it for media storage i.e. it's not being accessed constantly, if this is to be used as a file sharing option and being accessed regularly it's not a good idea.
    I noticed from your first post that you added the SnapRaid Parity drive to mergerfs, this is a no no, if you have 4 drives the largest should be used as your Parity drive with the other 3 as your mergerfs and SnapRaid data drives.

  • Please do not use USB drives in a RAID config. Only bad things will ensue.
    Read the very many forum posts on people running into issues with this config of RAID with USB drives.


    If anything shuck the drives out of the containers and use proper SATA connections.

  • Yes, using mergerfs (unionfs) you have a single mount point, but this is where new users get confused (I did), my mergerfs is set as MFS (Most Free Space) that means a file is copied to the drive with the most free space.

    Am I correct in thinking that MFS is based upon the % of the drive's free capacity as opposed to literally how may MBs each hard drive has spare?

  • Please do not use USB drives in a RAID config. Only bad things will ensue.
    Read the very many forum posts on people running into issues with this config of RAID with USB drives.


    If anything shuck the drives out of the containers and use proper SATA connections.

    Is this referring to snapraid or just standard raid setups? From what I have read / watched tutorial wise, I was under the impression that snapraid was OK with usb drives. Perhaps I am more confused than I realised lol.


    If I shuck the drives, and use proper sata connections - how best to connect these to my RPi 4?

  • There is a caveat to using mergerfs (unionfs) you cannot point docker or docker configs to mergerfs mount point it simply doesn't work!! you either have to point those to a single drive within the mount point or use a separate drive.

    And my confusion increases.... as I have deluge working as a docker contained app (I think this is the correct terminology, apologies if not!) which is downloading to / seeding from a folder one level down from the root of my mergerfs mount point. Or is that what you mean by 'single drive within the mount point'?


    I noticed from your first post that you added the SnapRaid Parity drive to mergerfs, this is a no no, if you have 4 drives the largest should be used as your Parity drive with the other 3 as your mergerfs and SnapRaid data drives.

    Yes, I did do this, as I was not clear whether this was needed or not. The third drive was set as 'parity only' in the snapraid config. I should remove it from the mergerfs config then? That would make more sense as I would see the correct amount of free space in the mount point, but wasnt clear (my own fault) if this would be accommodated for by the fact that in snapraid the two 'data' drives were set as data, no parity, and the one parity drive was set as such.


    Forgive me, not trying to sound difficult, just trying to wrap my head around it all!!!

  • Forgive me, not trying to sound difficult, just trying to wrap my head around it all!!!

    That's OK, I have a separate drive for my docker and docker configs, but a user recently was having a problem with docker configs on the mergerfs mount point, he overcome that by placing the configs on a drive within the mergerfs mount point and it worked.


    The third drive was set as 'parity only' in the snapraid config. I should remove it from the mergerfs config then?

    Yes you should, I have 4 drives, 3 for mergerfs + snapraid data and 1 for the snapraid parity

  • Well I think at this point it will be best for me to start over again with a clean slate. Before I do, however, can anyone please confirm or deny the following:

    Please do not use USB drives in a RAID config. Only bad things will ensue.
    Read the very many forum posts on people running into issues with this config of RAID with USB drives.


    If anything shuck the drives out of the containers and use proper SATA connections.

    Is this true for snapraid (in which case my setup is a waste of time)? Or is this only true for standard raid setups?


    Please let me know before I start starting again! lol


    Thanks for all the help

  • @Reishtak
    Your use description, choice of hardware and need(?) for extra redundancy, doesn't add up.


    If you require a low power storage solution, by all means use RPi4 and a single external USB disk. If you worry about data loss, backup the data over the network or to a separate (can be turned off or allowed to spin down) powered USB disk.


    If you also require reliability and performance, avoid introducing extra complexity. Make sure to use a single, possibly very big, fast and high quality (3-5 years warranty) drive in a good external powered enclosure. Don't use a cheap external USB drive intended for desktop or backup use. Don't use a USB hub.


    If you require redundancy to be able to provide very low downtime, then don't use a RPi4, don't use USB attached drives (except possibly for backups), don't use a USB hub.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Took me a while to find it but you can use USB on snapraid, a user posted this about his own set up using a Pi4

    Thanks :) Looks like just what I am after, and finally some real-world use information on RPi4 with OMV & Snapraid + USB hubs.

    Thanks for the input, but I'm not sure I understand what your suggestion in my situation really is...


    I am trying to adopt a low power solution, yes. However I can't simply go for a RPi4 and single USB disk, as the largest affordable disks are around 14TB - I already have this much data, which means that option is not viable.


    I don't have a need for EXTRA redundancy, I have a need for SOME redundancy (currently have basically none, either in the form of hardware failure, or in terms of proper backup) - from what I understand, snapraid seems to provide a mix of redundancy in terms of hardware failure (I currently have none) and a basic form of backups - (I currently manually back up some files very infrequently, which is a poor situation I wish to improve on).


    If I (as you suggest) backup the data over the network, this requires not just another very large single USB disk, but another host to run the disk from. Assuming this to be another RPi (do you have any suggestions for alternatives, with low power being a fairly high priority?) then my power usage doubles.


    I do not really require performance above what I believe the RPi 4 can provide (streaming HD files over wired network, hosting files etc). Reliability is important to me, but I believed the RPi4 would provide this - are there other better solutions (ideally with low power consumption as well?) - I have been running a RPi3 with PiHole / DHCP server for 12+months without a reboot....


    Downtime is an inconvenience for me but not a prime concern. Obviously I would rather not have down time but it is not a major issue.




    I'm not trying to argue with what you say, more trying to understand what better alternatives you can think of given my use description?


    Price per GB seems to be best on 8TB WD element external drives at the moment. While I am sure these can be shucked out of their enclosures, I am not sure that this makes sense for me if snapraid works OK on USB drives, as they will spin down, reducing power usage and can be run off the RPi4. The (admittedly basic) backup and redundancy features of snapraid seem to provide what I need, unless I am failing to grasp something fundamental (which is surely possible!!!)


    The link Geaves posted above seems to suggest what I am trying to achieve is, in fact, possible...?



    Cheers for the input all :)

  • OK. You don't need high performance. You don't need redundancy.


    You seem to think that snapraid is a form of backup? It isn't. Snapraid can in some special cases handle problems better than RAID, but you STILL need some form of backups if you want to avoid losing data. The adage "RAID is not backup" applies to Snapraid as well.


    So if you value your data you need backups. If you even consider your data to be important, you need more than one backup copy. I have at least four or five copies of some (not a lot) of my data. But most is only backed up once or twice.


    There is nothing stopping you from using several HDDs with a RPi4. I have a powered dual drive USB enclosure with 2x8TB HDDs connected to my RPi4. Works fine with 12TB HDDs, but not with 16TB HDDs for some reason. There are similar enclosures with up to 10 HDD bays. I would not recommend that you use a multiple bay USB enclosure for RAID or Snapraid, but it should be possible. I don't recommend it because I wouldn't trust it and/or performance would be low. I use my dual bay USB enclosure as JBOD with mergerfs for backups only. I consider getting another one.


    There is nothing stopping you from using multiple RPi4 or using up to two external USB HDD enclosures with each RPi4. To use more than two enclosures you would need to use some form of USB hub, and I wouldn't recommend that. Again trust and performance.


    I use multiple Odroid HC2s with 12TB or 16TB HDDs.


    When you consider the price for various size HDDs, don't just consider the cost of the HDD itself. Also add the cost needed to make the HDD available on the network. Include part of the cost of the NAS computer. Any external enclosure. Part of the network infrastructure. If you include these costs you may find, as I did, that 12TB or even 16TB HDDs are more cost effective than 8TB HDDs. As well as being more power efficient and require less labor.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

    Edited once, last by Adoby ().

  • Awesome, thanks - that clears a lot up.


    I was under the impression however that with snapraid you could perform file restores from each 'snap' so that with, for example, media storage (films, music, tv episodes etc) where files do not change very often, you can use snapraid as a form of restore if you, for example, accidentally delete files. Is that not correct? I am of course aware that off site backups etc are needed for proper safety.


    Most of my used storage space is this type of media, with some small amount being personal files which I can carry on backing up as I have been (albeit more frequently!) but on the whole it seems that my idea is not, fundamentally at least, flawed.


    Cheers for your advice on cost, does make sense & I'll consider all that when buying.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!