[HowTo] SnapRAID in OMV

    • Does one need to enable "Run Scrub - Set to true if you want to scrub after a successful sync" in order for the options "Scrub Frequency" and "Scrub Percentage" to take effect? The option sounds like it will run regardless of the set scrub frequency but the SnapRAID logs say "Array scrubbing is not enabled" so should it be enabled or not?
    • Hello all!

      is it a regular behavior of snapraid, that it runs the sheduled sync-job or the sheduled diff-script-job for exactly one hour and produces one notificationmail every single minute? I allways receave about 59 emails just because of snapraid. The result is, that I actually don't notice my notificationmails anymore...

      I've had this behavior already in OMV 2: Had a sheduled sync running, because I wasn't aware of the diff-script. After upgrade to OMV 4 I found the same behavior as I sheduled the sync like bevore and now - since i'm aware of the sence and function of the diff-script - i find in different style the same: 59 emails over one hour.

      I have only one sheduled snapraid job as far as i can see in Web-Gui and there was no snapraid-process running bevor the sheduled job started.

      Contend is almost the same in every email:

      Brainfuck Source Code

      1. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'OMV_MAIL_primarymail NAS_XXX.YYY@outlook.com'
      2. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'OMV_MAIL_secondarymail XXX.YYY@hotmail.com'
      3. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'OMV_CONTENT_FILES: /srv/dev-disk-by-label-Parity01/snapraid.content
      4. /srv/dev-disk-by-label-Daten02/snapraid.content
      5. /srv/dev-disk-by-label-ALT03/snapraid.content
      6. /srv/dev-disk-by-label-ALT04/snapraid.content
      7. /srv/dev-disk-by-label-ALT05/snapraid.content'
      8. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'PARITY_FILE: /srv/dev-disk-by-label-Parity01/snapraid.parity'
      9. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'OMV_SNAPRAID_DISKS: /srv/dev-disk-by-label-Daten02
      10. /srv/dev-disk-by-label-ALT03
      11. /srv/dev-disk-by-label-ALT04
      12. /srv/dev-disk-by-label-ALT05'
      13. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'TMP_OUTPUT: /tmp/snapraid/snapraid.out'
      14. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'RUN_SCRUB: true'
      15. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'SCRUB_FREQUENCY_IN_DAYS: 1'
      16. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'SCRUB_PERCENT: 5'
      17. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'SCRUB_OLDER_THAN_DAYS: '
      18. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'DEL_THRESHOLD: 100'
      19. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /srv/dev-disk-by-label-Parity01/snapraid.content'
      20. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /srv/dev-disk-by-label-Daten02/snapraid.content'
      21. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /srv/dev-disk-by-label-ALT03/snapraid.content'
      22. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /srv/dev-disk-by-label-ALT04/snapraid.content'
      23. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /srv/dev-disk-by-label-ALT05/snapraid.content'
      24. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: PARITY_FILE: /srv/dev-disk-by-label-Parity01/snapraid.parity'
      25. [2019-05-01 04:36:01] omv-snapraid-diff: INFO: 'SCRUB_OLDER_THAN_DAYS not set in '/etc/snapraid-diff.conf''
      26. [2019-05-01 04:36:01] omv-snapraid-diff: INFO: 'set it to 0'
      27. [2019-05-01 04:36:01] omv-snapraid-diff: INFO: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
      28. [2019-05-01 04:36:01] omv-snapraid-diff: INFO: 'SnapRAID Job started - Script-version: 1.0.8'
      29. [2019-05-01 04:36:01] omv-snapraid-diff: INFO: 'SnapRAID-Version: 11.3 by Andrea Mazzoleni, http://www.snapraid.it'
      30. [2019-05-01 04:36:01] omv-snapraid-diff: INFO: 'Checking 'disks' in '/etc/snapraid.conf''
      31. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /srv/dev-disk-by-label-Daten02'
      32. [2019-05-01 04:36:01] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /srv/dev-disk-by-label-ALT03'
      33. [2019-05-01 04:36:02] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /srv/dev-disk-by-label-ALT04'
      34. [2019-05-01 04:36:02] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /srv/dev-disk-by-label-ALT05'
      35. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: 'SnapRAID DIFF started'
      36. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: 'SnapRAID DIFF finished'
      37. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: '----------------------------------------'
      38. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: 'SUMMARY of changes since last sync:'
      39. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: 'Added: [0] - Deleted: [0] - Moved: [0] - Copied: [0] - Updated: [0]'
      40. [2019-05-01 04:36:02] omv-snapraid-diff: DEBUG: 'Pre-Hash option: --pre-hash'
      41. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: 'No change detected. Nothing to do'
      42. [2019-05-01 04:36:02] omv-snapraid-diff: INFO: 'SnapRAID Job ended.'
      Display All



      the first mail had an additional line after DEL_TRESHOLD: 100

      Source Code

      1. [2019-05-01 04:01:02] omv-snapraid-diff: INFO: ''/tmp/snapraid/snapraid.out' deleted - creating a new one'
      all other mails have equal content like quoted above.

      Strange is also, that I've done file-changes for testing. But EVERY email reports ZERO file-changes!

      Not so the /var/log/snapraid.log. It seems to mainly have the same content as the mails have (hard to exactly check) BUT will after 2 times of reporting ZERO changes at the beginning (Minute 01:04) report a

      Source Code

      1. [2019-05-01 04:02:33] omv-snapraid-diff: INFO: 'SUMMARY of changes since last sync:'
      2. [2019-05-01 04:02:33] omv-snapraid-diff: INFO: 'Added: [26] - Deleted: [0] - Moved: [4] - Copied: [0] - Updated: [6]'
      3. [2019-05-01 04:02:33] omv-snapraid-diff: DEBUG: 'Pre-Hash option: --pre-hash'
      4. [2019-05-01 04:02:33] omv-snapraid-diff: INFO: 'Changes detected [A-26,D-0,M-4,C-0,U-6] -> deleted files (
      5. T) is below threshold (100). Running SYNC Command.'
      6. [2019-05-01 04:02:33] omv-snapraid-diff: INFO: 'SnapRAID SYNC Job started'
      that corresponds to what I changed bevor AND would be the Information I'd like to see in one notificationmail. But i have not receaved a mail telling me that changes have been dedected and computed.


      After one hour I don't receave further mails and also the /var/log/snapraid.log gets no further entry but the snapraid-process still runs. Maybe because of the 5% scrub??


      edit:
      seems to be so...

      Brainfuck Source Code

      1. [2019-05-01 04:59:01] omv-snapraid-diff: INFO: 'SUMMARY of changes since last sync:'
      2. [2019-05-01 04:59:01] omv-snapraid-diff: INFO: 'Added: [0] - Deleted: [0] - Moved: [0] - Copied: [0] - Updated: [0]'
      3. [2019-05-01 04:59:01] omv-snapraid-diff: DEBUG: 'Pre-Hash option: --pre-hash'
      4. [2019-05-01 04:59:01] omv-snapraid-diff: INFO: 'No change detected. Nothing to do'
      5. [2019-05-01 04:59:01] omv-snapraid-diff: INFO: 'SnapRAID Job ended.'
      6. [2019-05-01 09:24:07] omv-snapraid-diff: INFO: '----------------------------------------'
      7. [2019-05-01 09:24:07] omv-snapraid-diff: INFO: 'SnapRAID SCRUB Job finished'
      8. [2019-05-01 09:24:07] omv-snapraid-diff: INFO: 'Full statusreport sent to '''
      9. [2019-05-01 09:24:07] omv-snapraid-diff: INFO: 'SnapRAID Job ended.'
      now the snapraid-process is also stopped (disappeared)

      would be nice to receave the "full statusreport" after successfull scrub instead of the 59 similar mails above! Is there any way to do that?
      /edit


      Anyone here to tell me if that is all normal or what i maybe did wrong?
      Thank you!

      The post was edited 3 times, last by Bausau ().

    • After obversation over a couple of days I have to add:


      there are Status-mails at the end after finishing the scrub-job!
      they obviousliy were delivered too late on my system due to unkown external reasons (provider unreachable, ore whatever)...



      1.) there's a Status-mail simply informing me about: [NAS.local] SnapRAID - Sync/Srub Job COMPLETED

      2.) there's further more a mail informing me about Data errors found during the job (the .xls problem in my case)

      3.) and there's a Status-mail containing all Information that has been already send in the again 59 mails before and a bit more (the amount of changed files for example!!!)



      So everything would be great if there's a possibility to stop the system sending 59 simmilar mails during exactly one hour after beginning the shedulled diff-scribt-job.
      I've read from a couple of poeple who complained about that the Statusmail is a bit spamy. I understood, that they talked about ONE mail containing too much Information.



      Does somebody have the same behavior with 59 mails in the first hour?
      Is there a way to stop that?



      Thank you!
    • Poincare1 wrote:

      My understanding is one drive can be content and data while the other can be parity.
      The typical setup is 1 parity drive to 2 or 3 data drives. However, what you're proposing will work but note,, without at least one copy of the content file, there's no recovery. So, if the content file is on the data disk only, and the data disk dies, there's no recovery.

      If you're going with 2 drives (1 data + 1 parity), you'll need to put a content file on the parity drive. In fact, it's a good idea to put a copy of the content file on all drives.
    • Poincare1 wrote:

      I have two 8TB drives that I would like to use with Snapraid. My understanding is one drive can be content and data while the other can be parity. That is how I believe they should be configured. As setup in Snapraid OMV plugin, does this seem like a good configuration?
      Like crashtest said, this will work but make sure you have a copy of the content file on both drives. Why do snapraid with only 2 drives? Do you plan to add additional drives in the future?
    • jollyrogr wrote:

      Poincare1 wrote:

      I have two 8TB drives that I would like to use with Snapraid. My understanding is one drive can be content and data while the other can be parity. That is how I believe they should be configured. As setup in Snapraid OMV plugin, does this seem like a good configuration?
      Like crashtest said, this will work but make sure you have a copy of the content file on both drives. Why do snapraid with only 2 drives? Do you plan to add additional drives in the future?

      Well, I have two 8TB drives and what I want to do is combine them, with hopefully some raid config as a backup. The data on one of the 8TB is already being backed up.

      I thought we could have 1 drive data and another as parity? Is there some better way in Snapraid, or another OMV plugin, to get it working?
    • Poincare1 wrote:

      Well, I have two 8TB drives and what I want to do is combine them, with hopefully some raid config as a backup. The data on one of the 8TB is already being backed up.

      I thought we could have 1 drive data and another as parity? Is there some better way in Snapraid, or another OMV plugin, to get it working?
      Having only two drives and wanting to "combine" them is inconsistent with SnapRaid, but this does depend on what you mean by "combine" which you are not making clear here.

      You can have one drive as data and one drive as parity on SnapRaid, but this doesn't "combine" them.
      --
      Google is your friend and Bob's your uncle!

      OMV 4.x - ASRock Rack C2550D4I - 16GB ECC - Silverstone DS380

      The post was edited 1 time, last by gderf ().

    • jollyrogr wrote:

      Why do snapraid with only 2 drives?
      I tested it one time, in a two drive setup, just to see if it would work. (It did.)
      The only reason I could come up with for using 2 disk SNAPRAID, would be for bit rot protection. Also SNAPRAID, and a simple filesystem like EXT4, will work reasonably well with USB connected drives.

      (Setting bit rot protection aside, which is a big deal in itself, creating a simple mirror with Rsync provides roughly the same benefits.)
    • crashtest wrote:

      jollyrogr wrote:

      Why do snapraid with only 2 drives?
      I tested it one time, in a two drive setup, just to see if it would work. (It did.) The only reason I could come up with for using 2 disk SNAPRAID, would be for bit rot protection. Also SNAPRAID, and a simple filesystem like EXT4, will work reasonably well with USB connected drives.

      (Setting bit rot protection aside, which is a big deal in itself, creating a simple mirror with Rsync provides roughly the same benefits.)
      I agree Crashtest. Bit rot protection would be the major benefit. Also with Snapraid it's very easy to add addtional data drives and parity drives in the future, so if 2 drives were only the beginning with future plans for expansion that makes sense.

      Are they USB drives? Personally I would not use USB unless I was fully backed up elsewhere and 100% OK with losing the data on the USB drives.
    • Poincare1 wrote:

      Well, I have two 8TB drives and what I want to do is combine them, with hopefully some raid config as a backup. The data on one of the 8TB is already being backed up.
      What platform are you running? Is it ARM or X86?

      A close RAID equivalent is SNAPRAID+UnionFS. Actually, this combination is superior to RAID5 in many ways, and includes bit rot protection.
      (BTW: The UnionFS plugin is actually mergerfs.)
      _______________________________________________________

      What is the storage requirement? How much do you need?
    • New

      AndrewZ wrote:

      What is the right way to override the default exclude list? Should this be done in /etc/default/snapraid?

      The path is /etc/snapraid.conf

      Universal excludes can be done there.
      ____________________________________

      It's also possible in the GUI as follows:



      (In this case, I chose a protected drive and excluded *.tmp files.)

      The post was edited 1 time, last by crashtest ().

    • New

      crashtest wrote:

      The path is /etc/snapraid.conf
      This file is auto generated and should not be edited - right?
      My goal was to override (or get rid of) default exclude *.bak - how can I do that?
      Thank you for pointing me to Rules tab, but that will not help in my case - a new user rule will be added after the existing system or default rules and according to the manual, "Previous rules have the precedence over the later ones"

      I can edit /usr/share/openmediavault/mkconf/snapraid and comment the unnecessary exclude line, but don't think it's the right way.

      The post was edited 1 time, last by AndrewZ ().

    • New

      AndrewZ wrote:

      how can I do that?
      You can't unless an include line would override the exclude. Never tried.

      AndrewZ wrote:

      I can edit /usr/share/openmediavault/mkconf/snapraid and comment the unnecessary exclude line, but don't think it's the right way.
      That is fine but the change will be overwritten by updates.
      omv 4.1.23 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!