Drive Pooling + Backup Strategy

    • OMV 3.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Drive Pooling + Backup Strategy

      Hey guys,

      I'm just getting started with OMV and am trying to figure out how to get the most (space) out of my built while still having a strong backup strategy.

      I've got 3 external 2.5" USB drives (3TB, 3TB, 2TB), 6 internal 3.5" drives (1TB, 1.5TB, 2TB, 3TB, 3TB, 3TB) and am running OMV 3.0.59 on a Raspberry Pi 3. I plan on using my built as media server for Plex and for TimeMachine backups for the Macs in my network.

      My original plan was to keep the external drives as main drives and just mirror them 1:1 to three internal drives with rsync, mirror two 3TB internals and use the rest for TimeMachine. I travel a lot and it would be nice to be able to just pull out one of the external drives and have a part of my media library with me. But now reading about drive pooling, snapraid, mergerfs etc. I'm not sure this is the smartest way to do this and quite frankly am getting a bit confused.

      What would you recommend? Having two pooled drives and mirror them or would I gain more space with snapraid and parity disk(s)? Or is there a completely different approach that I'm missing?

      I'd love to just have one big chunk of drive space but would still like to keep the possibility to take my external drives on the road with me. Is that doable?

      Any help is greatly appreciated!
      V
    • well you can sort of still have what you want and use SnapRaid and other.

      I am a bit confused on how you can have internal drives if you are running on Pi.

      I did not know you can connect SATA or other drives to it..

      However putting that aside,
      I would build out an array using internal disks , if you use SnapRaid you can get about between 7TB and 10TB of space

      1x3TB -- Parity + ( 1TB, 1.5TB, 2TB, 3TB, 3TB) -- data ~ 10TB
      2x3TB -- Parity + ( 1TB, 1.5TB, 2TB, 3TB) -- data ~ 7TB

      and using mergerFs on top will give you a single volume to share leaving all the data on individual disks

      than rsync to external drives as needed.

      this gives you a fully functional server and a backup on external drives.
      as/if your main storage grow you can upgrade the external drives as needed to accommodate new data.
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • Thanks for your help! And yeah sorry, it's just 3.5" disks with SATA to USB adapters sitting inside my case with the Pi.

      I like your solution, but if SnapRaid really does only needs two parity drives for up to 14 disks, I could also just keep one 3TB external as "travel drive" and just rsync whatever I want to take. I wouldn't take more than one drive on the road anyways. That way I'd gain 4TB of space. Regarding parity, are 2x3TB internals better/safer than to use 2x2TB?

      Union filesystems in OMV uses mergerFS to pool the drives, correct?

      The main reason for me wanting to use the 2.5" external disks as "main" disks was noise, with mergerFS can I somehow control what data "spills over" to which drive if one is full?
    • You really have 9 drives on a RPi3???

      Vandal wrote:

      Union filesystems in OMV uses mergerFS to pool the drives, correct?
      Yes.

      Vandal wrote:

      with mergerFS can I somehow control what data "spills over" to which drive if one is full?
      There are many different ways mergerfs can control the data. Read the policies section here.

      I would snapraid on that many drives on an RPi would take forever to sync. It also would slow down everything especially since you would be saturating the usb bus that the network adapter shares. I think you need to look into getting a real motherboard.
      omv 4.0.5 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.0.2
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • Ok, this is snapraid parity matrix.
      First number is number of parity disks ,last number is max number of related data disks
      So, you can have up to 4 data disk with 1 parity
      Up to 14 data disks with 2 parity. After that each parity disk adds about 6 or 7 data disks to the total.

      Parities Data disks
      1/Single Parity/RAID5 2 - 4
      2/Double Parity/RAID6 5 - 14
      3/Triple Parity 15 - 21
      4/Quad Parity 22 - 28
      5/Penta Parity 29 - 35
      6/Hexa Parity 36 - 42


      The only other must is that parity drives must be the largest drives in the set.
      I.e. if the largest data drive in the set is 3tb as is your case, than parity drives must be 3tb. Hence my proposed config.

      Unfortunately you need to sacrifice one or two of your 3tb drives for parity if you use snapraid or even some other data protection system.


      As for noise, using 2.5 drives as main drives will not help with noise much.
      In my experience 2.5 drives could be noisier than 3.5 once some times.
      I am not sure if you can controll spillover but you can controll what data hose where initially by putting appropriate folders on drives you want. Mergerfs will present the drives and folders as is. Only newer folders will be created using config settings.


      Sent from my SM-N910T using Tapatalk
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • Using an RPI as a NAS is a bad idea in general because it uses the same limited bus for LAN and all USB ports, but that many drives on an RPI is a recipe for disaster. Any kind of backup or syncing is going to be a performance nightmare.

      You need to get a proper motherboard with either a decent number of SATA ports or a slot for a SATA controller card. Then we can talk about configuring RAID or SnapRAID.
    • Hi,

      I have roughly the same need as you describe: storage for media, Plex MS, time machine backups for 2 Macs in the house.
      Since I don't have a NAS, yet, I was reading up on a lot of the stuff and here are my insights so far:

      1) as already mentioned, a Raspberry Pi is a bad idea a) for the number of disks you mention, b) for Plex

      2) I would keep my time machine HDDs out of the raid, one for each computer and without extra backup. It's already a backup and the deleteted "back-in-time" functionality is - at least for me - not so importand as to warrent an extra disk.

      3) I would keep the Plex library folder out of the snapraid because of the many small files and changes that are performed in the background. Since snapraid does not support a real time redundancy model, it will have a hard time at the end of the day figuring out what actually changed on the disks. Therefore, for me, putting OMV on a bootable USB stick is also out of the questions (OMV also discourages this due to underlying debian). I prefer to have the system on an 2.5" SSD. This will also speed up Plex's transcoding features. I would regularly clone the system drive (or parts of it) to either a USB-connected drive or as an archive to the snapraid.

      4) leaving the rest of the HDD to be used with snapraid for media and other seldomely changing files.

      Hope this helps.