Novice setting up home NAS - after opinions and help if possible!

Hi all, I am posting here hoping for a bit of advice, or direction towards guides which may answer my questions but I have missed. I've tried to read all I can and answer my questions myself, but I feel they are probably too specific to be covered and came here for some help!

What I am trying to achieve: I would like a low power consuming NAS, from which I can serve my files, run a media server, do some light torrenting, and (ideally) backup my important data.

My old setup: (Very) Old Dell optiplex, running Windows 10, with a JBOD setup of 5 data drives totaling 18TB (1 x 8TB, 1 x 5TB, 1 x 3TB, 2 x 1TB), all NTFS, shared on the network. Also ran plex and deluge from windows on this box. Some of the drives were very old and not in great shape (mainly the smaller three) and my only backup was a recently purchased 8TB USB3 HDD (which I would manually back up to fairly infrequently). Of the 18TB available, I would say that around 15TB was actually used, so I also needed to increase storage shortly.

So after lots of researching, I couldn't decide what would be the best solution for me. I have really only scratched the surface of linux before, but had some experience with raspberry pi usage (I am currently running a 3b+ with PiHole on it to reduce adverts at home), but once reading that the Pi 4 was pretty good for plex and as a NAS with the USB 3 etc, I decided to make some purchases and go from there, so I bought myself a Pi 4, 4GB model, a powered USB 3 hub, and two new 8TB USB3 HDDs, and started researching.

My current setup: Raspberry Pi 4, running OMV, with snapraid and unionfs, with my three 8TB USB3 HDDs connected through the powered hub to the Pi. One drive is dedicated to parity, the others are allocated for data. I have managed to get deluge and plex running through docker / portainer (although even when following guides for this I feel like its a bit hit and miss. Deluge seems to be stable now I have the permissions figured out, but I broke Plex again yesterday after configuring the snapraid/unionfs).

Some questions: 1. Do I need to (or should I, given my usage case) run unionfs as well as snapraid? The way I configured it was to set up the snapraid first (the guides I read seemed to say that it doesn't matter which way round you do this), then add the three drives (two data, one parity) in unionfs to show as one large storage, filling the drive with most free space first. a) was I wrong to add the parity drive in unionfs? b) is it even worth using unionfs in my usage case - it seems to me that equalling the drive usaage would be a good thing, but I am really confused here.
2. Is my setup sensible? I am currently copying data from my old drives to the new snapraid/unionfs share and once this is completed I hope to sync the snapraid. Once this is done, and I am happy that the data is all there, I was planning on adding some of my old drives to the setup. a) can I easily do this and should I wipe / format the drives? They are all NTFS and I presume that ext4 would be preferrable (the three 8TB drives I am using now are all wiped/formatted ext4 via OMV).

3. How risky is it playing around with the OMV setup when I have the data on my snapraid array? I figured from the reading I've done that it is fairly safe (which is why I and so many others are choosing it) but are there any big no-nos that I should be aware of? Docker, portainer etc??

4. Are there any optimisations I could perform? In reading as much as I can for this, there are quite a few topics that I've come across but not had much time to look into properly. For example, alternatives to plex - is emby any good and is it worth changing from plex (which I understand fairly well). Is it worth migrating my pihole setup to this raspberry pi?

I am sure that there are other questions I meant to ask and more that will pop up in the future, but honestly if you can provide any kind of opinion or help at all that would be massively appreciated.

Sorry for the huge post, but I figured more information was better than not enough!

Thanks in advance, and I look forward to learning from you guys.

Thanks for your reply. It's good to know that you have reinstalled the OS without any problems.

I am guessing from you saying that you have two mirrored drives that you are running a standard RAID (RAID1?).

Standard RAID does not really fit my use case as far as I can tell. Snapraid provides some form of backup, while allowing me to 'spend' much less HDD space on parity (or mirroring). I can still recover from hardware failures and as an added bonus have backups for things like accidental file deletions (if I am understanding it correctly).

I guess my most pressing question at the moment is: Do I gain anything by using unionfs as well as snapraid? If I mount a samba share on a snapraid array, won't it present as the whole available storage space, and will it not fill disks evenly?

What (if anything) will I lose out on by not using BOTH unionfs and snapraid? Is snapraid alone not recommended:?

Thanks

Zitat von savellm

Please do not use USB drives in a RAID config. Only bad things will ensue.
Read the very many forum posts on people running into issues with this config of RAID with USB drives.

If anything shuck the drives out of the containers and use proper SATA connections.

Is this referring to snapraid or just standard raid setups? From what I have read / watched tutorial wise, I was under the impression that snapraid was OK with usb drives. Perhaps I am more confused than I realised lol.

If I shuck the drives, and use proper sata connections - how best to connect these to my RPi 4?

Zitat von geaves

There is a caveat to using mergerfs (unionfs) you cannot point docker or docker configs to mergerfs mount point it simply doesn't work!! you either have to point those to a single drive within the mount point or use a separate drive.

And my confusion increases.... as I have deluge working as a docker contained app (I think this is the correct terminology, apologies if not!) which is downloading to / seeding from a folder one level down from the root of my mergerfs mount point. Or is that what you mean by 'single drive within the mount point'?

Zitat von geaves

I noticed from your first post that you added the SnapRaid Parity drive to mergerfs, this is a no no, if you have 4 drives the largest should be used as your Parity drive with the other 3 as your mergerfs and SnapRaid data drives.

Yes, I did do this, as I was not clear whether this was needed or not. The third drive was set as 'parity only' in the snapraid config. I should remove it from the mergerfs config then? That would make more sense as I would see the correct amount of free space in the mount point, but wasnt clear (my own fault) if this would be accommodated for by the fact that in snapraid the two 'data' drives were set as data, no parity, and the one parity drive was set as such.

Forgive me, not trying to sound difficult, just trying to wrap my head around it all!!!

Well I think at this point it will be best for me to start over again with a clean slate. Before I do, however, can anyone please confirm or deny the following:

Zitat von savellm

Please do not use USB drives in a RAID config. Only bad things will ensue.
Read the very many forum posts on people running into issues with this config of RAID with USB drives.

If anything shuck the drives out of the containers and use proper SATA connections.

Is this true for snapraid (in which case my setup is a waste of time)? Or is this only true for standard raid setups?

Please let me know before I start starting again! lol

Thanks for all the help

Zitat von geaves

Took me a while to find it but you can use USB on snapraid, a user posted this about his own set up using a Pi4

Thanks Looks like just what I am after, and finally some real-world use information on RPi4 with OMV & Snapraid + USB hubs.

Zitat von Adoby

@Reishtak
Your use description, choice of hardware and need(?) for extra redundancy, doesn't add up.

If you require a low power storage solution, by all means use RPi4 and a single external USB disk. If you worry about data loss, backup the data over the network or to a separate (can be turned off or allowed to spin down) powered USB disk.

If you also require reliability and performance, avoid introducing extra complexity. Make sure to use a single, possibly very big, fast and high quality (3-5 years warranty) drive in a good external powered enclosure. Don't use a cheap external USB drive intended for desktop or backup use. Don't use a USB hub.

If you require redundancy to be able to provide very low downtime, then don't use a RPi4, don't use USB attached drives (except possibly for backups), don't use a USB hub.

Alles anzeigen

Thanks for the input, but I'm not sure I understand what your suggestion in my situation really is...

I am trying to adopt a low power solution, yes. However I can't simply go for a RPi4 and single USB disk, as the largest affordable disks are around 14TB - I already have this much data, which means that option is not viable.

I don't have a need for EXTRA redundancy, I have a need for SOME redundancy (currently have basically none, either in the form of hardware failure, or in terms of proper backup) - from what I understand, snapraid seems to provide a mix of redundancy in terms of hardware failure (I currently have none) and a basic form of backups - (I currently manually back up some files very infrequently, which is a poor situation I wish to improve on).

If I (as you suggest) backup the data over the network, this requires not just another very large single USB disk, but another host to run the disk from. Assuming this to be another RPi (do you have any suggestions for alternatives, with low power being a fairly high priority?) then my power usage doubles.

I do not really require performance above what I believe the RPi 4 can provide (streaming HD files over wired network, hosting files etc). Reliability is important to me, but I believed the RPi4 would provide this - are there other better solutions (ideally with low power consumption as well?) - I have been running a RPi3 with PiHole / DHCP server for 12+months without a reboot....

Downtime is an inconvenience for me but not a prime concern. Obviously I would rather not have down time but it is not a major issue.

I'm not trying to argue with what you say, more trying to understand what better alternatives you can think of given my use description?

Price per GB seems to be best on 8TB WD element external drives at the moment. While I am sure these can be shucked out of their enclosures, I am not sure that this makes sense for me if snapraid works OK on USB drives, as they will spin down, reducing power usage and can be run off the RPi4. The (admittedly basic) backup and redundancy features of snapraid seem to provide what I need, unless I am failing to grasp something fundamental (which is surely possible!!!)

The link Geaves posted above seems to suggest what I am trying to achieve is, in fact, possible...?

Cheers for the input all

Awesome, thanks - that clears a lot up.

I was under the impression however that with snapraid you could perform file restores from each 'snap' so that with, for example, media storage (films, music, tv episodes etc) where files do not change very often, you can use snapraid as a form of restore if you, for example, accidentally delete files. Is that not correct? I am of course aware that off site backups etc are needed for proper safety.

Most of my used storage space is this type of media, with some small amount being personal files which I can carry on backing up as I have been (albeit more frequently!) but on the whole it seems that my idea is not, fundamentally at least, flawed.

Cheers for your advice on cost, does make sense & I'll consider all that when buying.