New Build planned - Many questions

  • Hi,


    I've been searching throughout the forums to gain a better understanding of OVM and the best way to configure it for my purposes. These questions span several topics, so I'm posting here in General Configuration discussions.


    Here are my goals at this point. I currently have a Windows server cobbled together for my media but I need a purpose built installation, and it's running out of space in the case.

    • This is to support my media for my Kodi installations, space for my photos in RAW format, and Documents.
    • I have PVR'd media and it will continue to grow.
    • I'd like to leverage the Crashplan plugin to backup my Photos and Documents, not my other media.
    • One SAMBA share for each media type is my ideal goal. PVR'd TV, Backed up Movies, Backed up Music, Documents, Books, and Photo's
    • I'd like to be able to survive 1 HDD failure, possibly two. I have had an HDD fail, not fun.

    I recently purchased a 12 bay case, 3 2.5's and 8 hot swaps on the front.


    I'm trying to understand file structure setup, parity using SnapRAID for redundancy, and adding new drives.


    I suspect the recommendation is EXT4, so let's set that as an assumption.


    Visualizing helps me, so I'm going to draw this out in a way here. Hopefully someone can use this to demonstrate structure and configuration. I'm a bit fried with 12-14 hours/7 days a week on an ERP project at work, so please be patient with me.


    Here's the physical drives. Let's say 1 120 GB for OVM then 6 of the 8 hot swaps filled.

    • Internal - sda0 - 120GB 2.5inch SSD for OVM/OS/Plugins
    • Hot Swap 1 - sda01 - 4TB
    • Hot Swap 2 - sda02 - 4TB
    • Hot Swap 3 - sda03 - 4TB
    • Hot Swap 4 - sda04 - 4TB
    • Hot Swap 5 - sda05 - 4TB
    • Hot Swap 6 - sda06 - 4TB
    • Hot Swap 7 - Empty
    • Hot Swap 8 - Empty

    So as I THINK I understand it, SnapRAID uses a parity disk. So instead of striping across all disks, we reserve 1 to maintain the parity bits.

    • Is this correct?
    • Is there a limit to the max number of drives for 1 parity drive.
    • Does that leave me with 20TB(5 Drives) of usable space?
    • Would anything be gained by adding another parity drive in the above scenario?

    File structure -

    • Is it possible to set up 1 share for each media type, then OVM manages it across the disks? Not sure of the terminology, virtual disks?
    • Or do I need to define shares on each drive. ie..

      • Assuming sda01 is dedicated parity

        • sda02

          • PVR'd TV
        • sda03

          • PVR'd TV
        • sda04

          • Backed Up Movies
        • sda05

          • Backed Up Movies
        • sda06

          • Photos
          • Audio
          • Documents

    That kinda cramps things if one category starts growing faster than expected.


    Adding drives. Is it as simple as putting in an additional drive and mounting it?


    Back to the setup above with the two empty Hot Swaps. I have my media sitting on a Windows Server today with NTFS. Is it possible/safe to take out each of those drives, plug it into an empty hot swap, then transfer the contents over to the new drive space? it would definitely be the fastest method.


    I'm in no rush, at least a month away from purchasing the rest of the hardware. I just want to attempt to set this up the best way possible for my situation on the first try.


    Thanks for your patience and guidance in advance.


    I'm dead tired, I probably won't be back on until the morning to respond. X/

    • Offizieller Beitrag

    Big question... Is the server going to run 24/7 with battery backup?


    Either way, you want to pool all your drives (software raid or unionfilesystem) and have redundancy (software raid or snapraid). Then you want to create a shared folder for each item you list originally (movies, docs, pics, etc). Then they can use whatever space is available. With six drives and allowing one drive to fail, you have 20 TB. With six drives and allowing two drives to fail, you have 16 TB. Adding a seventh drive will add 4 TB to the total of either option. Filesystem really isn't that important. ext4 will work fine.


    We can discuss the transfer after determining best setup. No matter what, you can put all the drives in the case after installing OMV.


    And one more thing... there are instructions to install crashplan on the forum but no plugin.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Zitat

    Big question... Is the server going to run 24/7 with battery backup?


    On the shopping list.


    Do you have any more info on an implementation of unionfilesystem in OVM? I'm guessing I would run with SnapRAID, then put a unionfilesystem on top of that? How does ufs manage writing data? So when I put a photo in the PHOTOS directory, ufs selects the physical drive to place it on or does it stripe? One of the benefits, as I understand it, of SnapRAID is that it only spins up the drive the file is on where other RAID solutions will stripe and spin them all up for read/write. So with SnapRAID, if you lose a drive, you only lose the files on it, not across all striped drives.

    • Offizieller Beitrag

    But is the server going to run 24/7?


    I was going to answer more based the hours the server ran but here is how snapraid and union filesystems (don't abbreviate as UFS because that is a different type of filesystem).


    snapraid works with existing filesystems like ext4 and xfs. It calculates parity to put on the parity drive. If a drive fails, you can use the little bits of info on the other drives with the parity to recreate the files on the failed drive. If more than one drive fails and you only have one parity drive setup in snapraid, you will just lose one drive worth of data. It has nothing to do with the union filesystems.


    Union filesystems like mhddfs, aufs, mergerfs (I use mergerfs) combine all the selected filesystems on data drives in a pool to make the drive look like one big disk. With most of them, when you write to the pool, it will write the files to the drive with the most free space but there are other options. When reading files, it only reads from the drive where the file is located. Union filesystems have nothing to do with snapraid.


    raid5 pools all drives and writes parity to all drives. If a drive fails, the info on it can be recreated from parity. If two drives fail, all data is lost.


    Combine snapraid (redundancy) and a union filesystem (pooling) and you get redundant, pooling storage. This is just like mdadm (software) raid except it is not real time like mdadm. If you haven't sync'd your snapraid parity and a drive fails, you may lose your new file created since your last sync. mdadm also spins all of the drives when reading or write. This uses more energy but is faster (typically).


    There are pros and cons of all of the options above. For my main server running 24/7 with battery backup, I use mdadm raid 5 on the disks. For my backup server, I have a mergerfs (union filesystem) pool - no snapraid. I also backup my files to LTO6 tapes.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • ryecoaaron - Thank you SO MUCH for clear and concise answers, as well as your patience with my questions. I'm going to dig into mdadm a bit as well. I will have more questions, I'm sure.


    I'll be back when I can focus, too much going on with work today to delve into all of it and think straight.

  • It's a good day!


    Machine is setup and OMV is installed. I've installed OMV-Extras as well. I haven't mounted the 4 - 4TB drives yet...


    So next steps, I'm guessing.

    • Mount the data drives
    • Configure Snapraid

      • Select Parity Drive
      • Then Content on one drive + data
      • Data on the other two.
    • Then install and configure UnionFilesystem?
      • Setup shares via UnionFilesystem? Or OMV will see only 1 drive for data at that point and set up my shares that way?

    Are there any recommendations/best-practice folder structures people use?

    • Offizieller Beitrag

    content should be on every data drive.


    Yes, I would use mergerfs and share that filesystem for everything. Too many different folder structure option. Do what works for you.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Nevermind, only data. I see how it works now!!! This is damned awesome!!!


    Small donation coming! I'll have to make it a habit to donate regularly so the devs can keep up the great work and to help offset hosting costs.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!