[HowTo] SnapRAID in OMV

    • Offizieller Beitrag

    That doesn't inspire me for either one :(

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    I haven't used it but the whiteout problem isn't a problem for you?

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I am very pleased to finally see not only Greyhole ported to OMV 0.5 but also SnapRAID. I have actually been using both of these programs with OMV 0.3 for months now and I was putting off updating my OMV server until at least MySQL and Greyhole were ported. Thanks to all who helped to bring this very useful functionality to an already great product.


    However, while running through a basic setup of the new SnapRAID plugin on an OMV 0.5 VM, I noticed that there is a software check that prevents one from placing a copy of the content file on a parity disk. Is there a specific reason for the plugin to impose this restriction or can this check be removed?


    From the online SnapRAID manual:

    Zitat

    The list of files is saved in the "content" files, usually stored in the data, parity or boot disks. These files contain the details of your backup, with all the checksums to verify its integrity. The "content" file is stored in multiple copies, and each one must be in a different disk, to ensure that in even in case of multiple disk failures at least one copy is available.


    I currently have an OS drive, 3 pooled data drives (using Greyhole), and a single parity drive. I prefer to use my pool drives for data only and so I keep copies of my content files on my OS drive and my parity drive. I understand there is some concern with running out of space on the parity drive, but this can be mitigated by use of the exclusion list rules.



    On a side note,

    Zitat von "viald"

    According to me Greyhole is not the best choice, because you don't obtain a virtual file system but a CIFS share which of course it's not view by the system the same way.
    For example if you want to add a path to your Plex configuration, I think that you can't do it using greyhole, because Plex lists only all file systems even if they are virtual.


    FYI, in my experience Greyhole has worked just fine with a Plex, minidlna, or MythTV server on the same machine. Greyhole uses a CIFS/SMB share which in turn is tied to a shared folder (aka landing zone) in the file system. Greyhole then maintains that shared folder with symlinks to the actual files in your pool after moving files out into the pool. As long as there are not any issues with following symlinks then just point whatever program (Plex, etc.) to the shared folder that is associated with your Greyhole CIFS/SMB share. Note that this method is primarily for READ ONLY access since writes directly to the shared folder are not processed by Greyhole. Additionally, having my Greyhole "landing zone" on an SSD essentially acts as a cache drive to greatly speed up writes to my pool (ie. I am only limited by either the source medium or gigabit ethernet speeds) all while letting individual drives spin down when idle to save power, etc.


    A better option is probably to locally mount the CIFS/SMB share directly. One quirk of directly accessing the shared folder is that creating new files doesn't seem to trigger Greyhole to properly copy them into the actual pool. A simple check for orphaned files in Greyhole will trigger a "cleanup", but this essentially introduces a manual or scheduled secondary process. As an alternative, the link below describes a quick way to mount a Greyhole share locally, which I plan to do while migrating my OMV from 0.3 to 0.5.


    https://github.com/gboudreau/G…e/wiki/MountSharesLocally

    Zitat

    Mounting your Samba shares locally is useful when you are using Greyhole, and want to write or in any way work with those files locally. Greyhole data should only be accessed through shares, so mounting those shares locally is an easy way to work with Greyhole data safely.

    • Offizieller Beitrag
    Zitat von "Hellspawn"

    However, while running through a basic setup of the new SnapRAID plugin on an OMV 0.5 VM, I noticed that there is a software check that prevents one from placing a copy of the content file on a parity disk. Is there a specific reason for the plugin to impose this restriction or can this check be removed?


    This is my mistake. It is difficult to get everything right when creating a plugin for something you have never used :) I will fix it.


    Zitat von "Hellspawn"


    There is a script called mountsmb-locally in /etc/init.d. I didn't find anything that activated it in the plugin but I left it anyway. I imagine it just needs to be started at startup??

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Content can go on the parity drive in the plugin now - version 0.5.11.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hi everyone,


    After some reading it appears as though snapraid would be better for me than RAID 5, pity I didnt read this sooner as I have just started a RAID 5 creation. I have two questions.


    1) Can I safely stop the RAID 5 creation and start again as snapraid, and if so how?
    2) How do I change from RAID 5 to snapraid?


    I'm a beginner when it comes to this so please guide me like a child :)


    My setup
    OMV 0.5
    HP Microserver N40L
    4x 3TB drives as the data drives
    250gb drive as the OS drive


    Main use is to share movies, videos, music etc. with the odd backup

    • Offizieller Beitrag

    Delete the raid and format each drive individually. Then add to snapraid.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Zitat von "ryecoaaron"

    Delete the raid and format each drive individually. Then add to snapraid.


    Thanks for the reply. I take it then I cant stop the raid creation while its on the go?


    Also forgive my noobness but can I then also use the pooling solution as described on the snapraid website?
    I would have have a movies folder on each drive but i would like my media player to see them as one large single movies folder. Most likely as a samba share.

    • Offizieller Beitrag

    You can stop it from the command line (as root) - md127 may have to be changed for your raid (look in raid tab):


    mdadm --stop /dev/md127
    mdadm --remove /dev/md127


    The pooling feature isn't enabled in the plugin because there is no way to share it. I am planning a pooling plugin but it may be a few weeks until it is ready.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Zitat von "ryecoaaron"

    You can stop it from the command line (as root) - md127 may have to be changed for your raid (look in raid tab):


    mdadm --stop /dev/md127
    mdadm --remove /dev/md127


    The pooling feature isn't enabled in the plugin because there is no way to share it. I am planning a pooling plugin but it may be a few weeks until it is ready.


    Thank you very much for the information and your help. If I can't pool the data (yet) then I will stick to RAID as the main reason for me to have a NAS is so that it looks like I have one folder as a source of media. When the pooling plugin is available then I will change over to that. Unless I can mimic that using the logical volume management?


    As a note, from http://snapraid.sourceforge.net/manual.html it says:



    Would that not work for me?

    • Offizieller Beitrag

    That works just fine but OMV will overwrite the configuration if you manually edit the file. I just of a way that might add it to samba.


    By the way, you are accessing your media over samba? You could always use plex or minidlna to "pool" the media.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I believe sharing the SnapRAID pool should be as simple as setting the pool path in your snapraid.conf to match that of a shared folder used by a SMB/CIFS share previously created through OMV. At least I think thats how I got around having to fight with OMV overwriting your config file. The pool command simply creates symlinks to what is in your pool and as long as SMB is set to follow symlinks then all should be well.


    I have basically tried this once before and it seemed to work ok from what I recall. The biggest drawback was that the pool command needs to be issued any time your pool drive contents change, otherwise your symlinks are outdated. This adds a manual/scheduled process and takes away from the usefulness of the feature. Additionally, a SMB/CIFS share of the SnapRAID pool is safe to use as read only, but writes may be a gamble since this does not give one sufficient control over where new files are actually stored in the pool drives. This is why I instead prefer to use Greyhole to manage the pooling side and SnapRAID to protect my data. This creates an extremely flexible but still robust NAS storage environment.


    As for RAID 5, it does nothing to prevent silent corruption, a.k.a. bit rot which is almost guaranteed to occur, for example, during a rebuild of an array of your size (see http://www.zdnet.com/blog/stor…stops-working-in-2009/162). LVM is a bad idea because loss of a single member drive will kill the whole array if I recall correctly. Furthermore, neither RAID5 nor LVM provide a way to recover accidentally deleted, corrupted, or modified files. Greyhole can add a sort of recycle bin to quickly recover accidentally deleted files, but this must be periodically emptied to recover drive space. SnapRAID can also restore both accidentally deleted files as well as files that have been changed/corrupted by rolling back to the version stored in the most recent sync snapshot.

    • Offizieller Beitrag

    I am changing snapraid to add its own samba share for the pool just like greyhole adds samba shares. I will also add a button in the commands tab to execute the pool statement. All of this is why I am working on an aufs plugin which would be a read/write pool. Then you wouldn't need the snapraid pool.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Zitat von "ryecoaaron"

    That works just fine but OMV will overwrite the configuration if you manually edit the file. I just of a way that might add it to samba.


    By the way, you are accessing your media over samba? You could always use plex or minidlna to "pool" the media.


    Still a noob here :)


    Basically I want to add a single "Movies" folder to my Mede8er and I belive my options are NFS or SMB, I decided on Samba so that I can access it from my Windows 7 PC. If I can do that with plex or minidlna then that would be great without worrying about losing the whole array if 1 or 2 drives fail. I'm actually beginning to think it best to forgoe parity and have seperate external backups, so I only need a way to "pool" everything into one "Movies" folder, one "Series" folder etc. I read up on LVM and discovered that again if one drive fails then the whole volume is lost. I'd prefer to only have to restore data one one drive than sit for 3 days rebuilding an array or copying everything over again. That is why snapraid seemed to be a better option.


    If plex or minidlna fill the bill it would be awesome but I have zero experience with them. (Though I believe Plex is only compatible with a plex client?)

    • Offizieller Beitrag

    If your Mede8er can access a dlna server (most media boxes can except roku), then plex and minidlna will work just fine. They pool the shares you have setup together.

    omv 7.0-32 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.9 | compose 7.0.9 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Zitat von "ryecoaaron"

    If your Mede8er can access a dlna server (most media boxes can except roku), then plex and minidlna will work just fine. They pool the shares you have setup together.


    Thanks, guess I'll give it a try. Thank you for the help, it is much appreciated.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!