Some questions about Snapraid...still learning.

  • Hi,
    I build my first snapraid-setup with unionFS in OMV.


    It seems to run OK.
    I have some questions...sorry for this.


    After the setup I am not sure if this is right...???
    I read it somewhere here in the forum that it should be done like this:


    One of my main questions is how to do the maintenance of my setup.
    What do I have to do after setup?


    If I get the status of the Raid I get this:



    ***
    No sync is in progress.
    The 100% of the array is not scrubbed.
    You have 443439 files with zero sub-second timestamp.
    Run the 'touch' command to set it to a not zero value.
    No rehash is in progress or needed.
    No error detected.
    ***


    What does all this mean?
    What is the touch command? I do not find it in the GUI. Should I do it in terminal? I fear to mess-up something there.


    What should I setup in the Settings tab of the plugin?



    If you have some links to read or any other advice it would be nice.


    Thanks for the help.
    F.

    MoBo: Fujitsu D3417-B
    CPU: Intel Celeron G3900
    RAM: Samsung DDR4 - 8 GB ECC
    Case: Fractal Design Node 80
    HDD: SSD 64GB (System), 2*3TB+1*4TB (Data) + 1*4TB (Parity). UnionFS + Snapraid
    OMV 4
    Router: Fritzbox 7590

  • I suggest reading thru the SnapRAID home page:


    http://www.snapraid.it/manual.html


    The script settings you have are reasonable. But these only affect the OMV provided script which I do not use. I prefer this one which has to be modified by editing it:


    https://gist.github.com/mtompkins/91cf0b8be36064c237da3f39ff5cc49d


    You can run this script as root by hand or schedule it. I have it scheduled to run every other day.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

  • OK, this looks promising.
    But I probably need help to set it up. First question comes to mind...where to save the script to not let it mess up during the next update of OMV?


    But I have to sleep now - it is in the middle of the night.


    Thanks for now.


    Best.
    Fabian

    MoBo: Fujitsu D3417-B
    CPU: Intel Celeron G3900
    RAM: Samsung DDR4 - 8 GB ECC
    Case: Fractal Design Node 80
    HDD: SSD 64GB (System), 2*3TB+1*4TB (Data) + 1*4TB (Parity). UnionFS + Snapraid
    OMV 4
    Router: Fritzbox 7590

  • OK, this looks promising.
    But I probably need help to set it up. First question comes to mind...where to save the script to not let it mess up during the next update of OMV?

    It's arbitrary where you put the script. But /usr/local/scripts could be considered customary.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    (While a bit redundant - a repost at the request of the user.)
    __________________________________________________________


    If your parity disk is the largest (or the same size) as the other two disks, you have the setup right, but you don't need a content file on the parity disk. I'd edit the parity drive and remove content. The rest is OK.



    Part of how SNAPRAID determines if a file has changed is the time stamp. Since it's unlikely to see files with duplicate time stamps with sub-second values, the "touch command" changes sub-second zeros to a value. The command for this is:


    snapraid touch


    This command only changes the sub-second portion of time stamps with zero values.
    _________________________________________________________


    To set up SNAPRAID for the first time, you need to run the command;


    snapraid sync


    Both commands can be automated and executed in Scheduled, Tasks. I run the following once a week, on Sunday at 00:15AM.
    (What you schedule, the interval, etc., is a matter of choice to fit your use case.)



    The full command line, in the above is:


    snapraid touch; snapraid sync


    (The semicolon runs the second command, after the first command, regardless of of whether or not the first command is successful.)


    The above will get you started and protected.
    ____________________________________________


    You can run many of snapraid's functions, manually, in the GUI. If desired, other commands you may want to use can be automated, in a similar manner, as shown above.
    To get a better idea of SNAPRAID functionality, how to restore, fix bitrot, etc., take a look at the SNAPRAID manual.

  • @flmaxey and @gderf thanks for your help.
    After touch and sync I get this now in status:


    ***
    No sync is in progress.
    The 100% of the array is not scrubbed.
    You have 14691 files with zero sub-second timestamp.
    Run the 'touch' command to set it to a not zero value.
    No rehash is in progress or needed.
    No error detected.
    ***


    So my questions:
    What is it about the scrubb?
    Why are still 14691 files with zero sub-second timestamp?


    Will all this be fixed if I use the script mentioned by @gderf?


    By the way, I changed the setup and removed the content from my paritydrive.


    But I will read the manual now...this will probably help ;)


    Thanks.
    Fabian

    MoBo: Fujitsu D3417-B
    CPU: Intel Celeron G3900
    RAM: Samsung DDR4 - 8 GB ECC
    Case: Fractal Design Node 80
    HDD: SSD 64GB (System), 2*3TB+1*4TB (Data) + 1*4TB (Parity). UnionFS + Snapraid
    OMV 4
    Router: Fritzbox 7590

  • I suggest configuring to place a content file everywhere that is offered. I can not think of a good reason not to do this.


    The script I suggested will touch files if needed and also run a scrub.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    2 Mal editiert, zuletzt von gderf ()

    • Offizieller Beitrag

    What is it about the scrubb?
    Why are still 14691 files with zero sub-second timestamp?

    New or open files (temp files for example) may need to be "touched" again. (But you went from over 400K to 14K files.)
    This is why automating it, (daily, whatever works for you) makes sense. These processes use resources so, once you're familiar with them, setting automated tasks for after-hours is the best practice.
    ________________________________________________________________


    On the Scrub:
    SNAPRAID scans files and assigns a checksum hash to them. A scrub rechecks the original checksums against file content, to see if there's a difference. Any differences are logged and are available using the command snapraid status.
    Section 4.1 of the SNAPRAID manual explains the order of commands from "Scrub" to "Fix". I'd run them manually a few times to see what they do, how long it takes, etc., before trying to automate them. (BTW, if you fill in System, Notification information and toggle on send E-mail report on, in Scheduled Jobs, you could get an E-mail of the output of the snapraid status command.)
    _______________________________________________________________


    (In being honest, since I have yet to finish configuring my SNAPRAID box, so I haven't looked over OMV's script to see what it does automatically, if anything. It's worth taking a look at, when you have the time, along with comparing it to @gderf 's script on github.)

  • I'd edit the parity drive and remove content. The rest is OK.


    I suggest configuring to place a content file everywhere that is offered. I can not think of a good reason not to do this.


    The script I suggested will touch files if needed and also run a scrub.

    Hm, 2 experts - 2 opinions...what to do?
    Can both of you explain your view?
    Thanks.


    I am still figuring out the rest - I will come back about this later. Some work to do...


    Thanks.
    F.

    MoBo: Fujitsu D3417-B
    CPU: Intel Celeron G3900
    RAM: Samsung DDR4 - 8 GB ECC
    Case: Fractal Design Node 80
    HDD: SSD 64GB (System), 2*3TB+1*4TB (Data) + 1*4TB (Parity). UnionFS + Snapraid
    OMV 4
    Router: Fritzbox 7590

  • It is impossible to recover a lost disk without a copy of the content file. Having more than one copy available hurts nothing and takes up a negligible additional amount of disk space.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    My reasoning;


    First we'll assume 3 drives; 2 data and 1 parity drive.


    - If one of the two data drives die, it takes at least one good copy of the content file to do a drive restoration. A good copy exists on the remaining, functional, drive.
    - If the parity drive dies, data is not lost. It can't be restored, just replaced. (Where a SYNC command regenerates it.)
    _____________________________________________________


    There's one exception to the above that I know of, if the Dev's haven't changed it.
    It's possible to run SNAPRAID with 2 drives and since SNAPRAID can fix bit-rot, arguably, it's far better than RAID1.
    In this case, to regenerate a single data drive, a content file would be needed on the parity drive.
    _____________________________________________________


    With the above noted, having a copy of the content file on the parity drive won't hurt anything. There's nothing wrong with putting a content file on all drives.

  • Suppose you have two data disks and want to be protected by SnapRAID in the case of both data disks failing simultaneously. For this to work you need a minimum of two parity disks. So far so good.


    However, if you did not place a copy of content file on any of the parity disks, and both data disks do simultaneously fail.................your data is unrecoverable.


    Note that the OMV SnapRAID plugin does not allow the system drive to hold a copy of the content file.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    The reference case, in this thread, is 2 data disks and 1 parity drive.


    Setting that aside, you have to admit 2 data disks + 2 parity drive is an unlikely SNAPRAID scenario for a beginner, and the simultaneous failure of 2 disks is unlikely. But in taking more scenarios into account, as previously mentioned, there's nothing wrong with setting a content file on the parity disk. And you're right about the content file being reasonable in size, relatively speaking. (I've done a two drive setup before.)
    ___________________________________________________________


    Along the lines of recovery, are you planing on setting backup @FFrank ? No fault tolerant setup is perfect. Host hardware and their power supplies fail, sometimes in an ugly way. The only way to insure that you don't lose data is to replicate it to another host or, possibly, to an external drive.

  • Setting that aside, you have to admit 2 data disks + 2 parity drive is an unlikely SNAPRAID scenario for a beginner, and the simultaneous failure of 2 disks is unlikely.

    Unlikely scenario or not doesn't get your data back.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

  • With the above noted, having a copy of the content file on the parity drive won't hurt anything. There's nothing wrong with putting a content file on all drives.


    Along the lines of recovery, are you planing on setting backup @FFrank ? No fault tolerant setup is perfect. Host hardware and their power supplies fail, sometimes in an ugly way. The only way to insure that you don't lose data is to replicate it to another host or, possibly, to an external drive.

    Hi and a big thanks to clarify this. This is a big help.
    So I will put a copy of the content file on all 3 drives - who knows when I will need it ;)


    Of course I plan to have an external backup. I use USB Backup to an external drive.


    I think I am quite OK with this plan?


    I am thinking about off-site backup in the cloud. But my internet-connection is quite slow and it will take ages to backup once...so I think I will skip this...


    But in case I will do it anyway...is there a nice plugin/docker to do cloud backup on a dropbox with encryption?


    Thanks again for your help @flmaxey and @gderf!!!


    Best.
    Fabian

    MoBo: Fujitsu D3417-B
    CPU: Intel Celeron G3900
    RAM: Samsung DDR4 - 8 GB ECC
    Case: Fractal Design Node 80
    HDD: SSD 64GB (System), 2*3TB+1*4TB (Data) + 1*4TB (Parity). UnionFS + Snapraid
    OMV 4
    Router: Fritzbox 7590

    • Offizieller Beitrag

    I keep my files local so I can't help with cloud backup.
    ___________________________________________________


    Note that various SNAPRAID disk-keeping operations can be automated without leaving the GUI.
    This can be done in, Scheduled Tasks.


    For example, I use the following (which may or may not be right for you):


    snapraid touch; snapraid sync
    (The above is a compound command. Search on-line to see what ";" and "&&" do in a command line.)


    In a similar manner, you can use Scheduled Tasks to run snapraid scrub -p 100 -o 20 and snapraid fix -e . (See the manual for the switch options used.) Finally, if you have System, Notifications setup and a scheduled task with snapraid status with the send e-mail switch on, you'll get an E-mailed report.

    • Offizieller Beitrag

    I am thinking about off-site backup in the cloud. But my internet-connection is quite slow and it will take ages to backup once...so I think I will skip this...


    But in case I will do it anyway...is there a nice plugin/docker to do cloud backup on a dropbox with encryption?

    I use Duplicati for this.
    There is a plugin and a docker. You can also install from CLI on Debian.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!