[HowTo] SnapRAID in OMV

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Not sure why. It seems to work on my system. The changes I made should execute the sync no matter what your setting is. Can you enable debug mode on the settings tab and post the output?
      omv 4.1.12 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • here you go

      Brainfuck Source Code

      1. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'OMV_CONTENT_FILES: /media/a3934a90-3256-466e-8a14-6782ec50d72e/snapraid.content
      2. /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87/snapraid.content
      3. /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.content
      4. /media/7f1e3c5e-31e5-4013-b066-dc7fd4bd235e/snapraid.content'
      5. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'PARITY_FILE: /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.parity'
      6. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'OMV_SNAPRAID_DISKS: /media/a3934a90-3256-466e-8a14-6782ec50d72e
      7. /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87'
      8. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'TMP_OUTPUT: /tmp/snapraid/snapraid.out'
      9. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'RUN_SCRUB: true'
      10. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'SCRUB_FREQUENCY_IN_DAYS: 7'
      11. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'SCRUB_PERCENT: 10'
      12. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'SCRUB_OLDER_THAN_DAYS: '
      13. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'DEL_THRESHOLD: 50'
      14. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/a3934a90-3256-466e-8a14-6782ec50d72e/snapraid.content'
      15. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87/snapraid.content'
      16. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.content'
      17. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/7f1e3c5e-31e5-4013-b066-dc7fd4bd235e/snapraid.content'
      18. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: PARITY_FILE: /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.parity'
      19. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'SCRUB_OLDER_THAN_DAYS not set in '/etc/snapraid-diff.conf''
      20. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'set it to 0'
      21. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
      22. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'SnapRAID Job started - Script-version: 1.0.8'
      23. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'SnapRAID-Version: 11.0 by Andrea Mazzoleni, http://www.snapraid.it'
      24. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'Checking 'disks' in '/etc/snapraid.conf''
      25. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /media/a3934a90-3256-466e-8a14-6782ec50d72e'
      26. [2017-03-13 00:12:47] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87'
      27. [2017-03-13 00:12:47] omv-snapraid-diff: INFO: 'SnapRAID DIFF started'
      28. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'SnapRAID DIFF finished'
      29. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: '----------------------------------------'
      30. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'SUMMARY of changes since last sync:'
      31. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'Added: [2] - Deleted: [0] - Moved: [0] - Copied: [0] - Updated: [0]'
      32. [2017-03-13 00:12:49] omv-snapraid-diff: DEBUG: 'Pre-Hash option: --prehash '
      33. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'Changes detected [A-2,D-0,M-0,C-0,U-0] -> deleted files (
      34. T) is below threshold (50). Running SYNC Command.'
      35. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'SnapRAID SYNC Job started'
      36. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: '----------------------------------------'
      37. Unknown option '?'
      38. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: '----------------------------------------'
      39. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'SnapRAID SYNC Job finished'
      40. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'SnapRAID SCRUB-Cycle count (7) not met (3). No scrub was run.'
      41. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'Full statusreport sent to '/var/mail/root''
      42. [2017-03-13 00:12:49] omv-snapraid-diff: INFO: 'SnapRAID Job ended.'
      Display All
    • I tried disabling the prehash button and then the cron job works

      Brainfuck Source Code

      1. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'OMV_CONTENT_FILES: /media/a3934a90-3256-466e-8a14-6782ec50d72e/snapraid.content
      2. /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87/snapraid.content
      3. /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.content
      4. /media/7f1e3c5e-31e5-4013-b066-dc7fd4bd235e/snapraid.content'
      5. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'PARITY_FILE: /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.parity'
      6. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'OMV_SNAPRAID_DISKS: /media/a3934a90-3256-466e-8a14-6782ec50d72e
      7. /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87'
      8. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'TMP_OUTPUT: /tmp/snapraid/snapraid.out'
      9. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'RUN_SCRUB: true'
      10. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'SCRUB_FREQUENCY_IN_DAYS: 7'
      11. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'SCRUB_PERCENT: 10'
      12. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'SCRUB_OLDER_THAN_DAYS: '
      13. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'DEL_THRESHOLD: 50'
      14. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/a3934a90-3256-466e-8a14-6782ec50d72e/snapraid.content'
      15. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87/snapraid.content'
      16. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.content'
      17. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: SINGLE_CONTENT_FILE: /media/7f1e3c5e-31e5-4013-b066-dc7fd4bd235e/snapraid.content'
      18. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: PARITY_FILE: /media/3c4e6274-be6d-4ee7-8ea3-f86c2fdc14d2/snapraid.parity'
      19. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'SCRUB_OLDER_THAN_DAYS not set in '/etc/snapraid-diff.conf''
      20. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'set it to 0'
      21. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
      22. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'SnapRAID Job started - Script-version: 1.0.8'
      23. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'SnapRAID-Version: 11.0 by Andrea Mazzoleni, http://www.snapraid.it'
      24. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'Checking 'disks' in '/etc/snapraid.conf''
      25. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /media/a3934a90-3256-466e-8a14-6782ec50d72e'
      26. [2017-03-13 00:18:03] omv-snapraid-diff: DEBUG: 'found: OMV_DISK: /media/d7962c63-c6ee-4cce-85d1-da3bac1bcb87'
      27. [2017-03-13 00:18:03] omv-snapraid-diff: INFO: 'SnapRAID DIFF started'
      28. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: 'SnapRAID DIFF finished'
      29. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: '----------------------------------------'
      30. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: 'SUMMARY of changes since last sync:'
      31. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: 'Added: [2] - Deleted: [0] - Moved: [0] - Copied: [0] - Updated: [0]'
      32. [2017-03-13 00:18:05] omv-snapraid-diff: DEBUG: 'Pre-Hash option: '
      33. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: 'Changes detected [A-2,D-0,M-0,C-0,U-0] -> deleted files (
      34. T) is below threshold (50). Running SYNC Command.'
      35. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: 'SnapRAID SYNC Job started'
      36. [2017-03-13 00:18:05] omv-snapraid-diff: INFO: '----------------------------------------'
      37. [2017-03-13 00:18:45] omv-snapraid-diff: INFO: '----------------------------------------'
      38. [2017-03-13 00:18:45] omv-snapraid-diff: INFO: 'SnapRAID SYNC Job finished'
      39. [2017-03-13 00:18:45] omv-snapraid-diff: INFO: 'SnapRAID SCRUB-Cycle count (7) not met (4). No scrub was run.'
      40. [2017-03-13 00:18:45] omv-snapraid-diff: INFO: 'Full statusreport sent to '/var/mail/root''
      41. [2017-03-13 00:18:45] omv-snapraid-diff: INFO: 'SnapRAID Job ended.'
      Display All
    • Looking at the debug output, the parameter is being set correctly.

      According to the manual, you can use -h or --pre-hash in the form snapraid -h sync or snapraid --pre-hash sync. The button on the drives tab actually uses snapraid sync -h which is the format that the snapraid binary tells you to use.

      I changed it to what the binary says in 3.6.4 in the repo now.
      omv 4.1.12 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • jollyrogr wrote:

      ahh. Looks like maybe you were missing the "-" in "--pre-hash"?
      Thanks. Didn't even notice that for some reason. Fixed in 3.6.5.
      omv 4.1.12 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Still having an error here on my fresh install (OMV 3.x and Snapraid 3.6.4 plugin

      Source Code

      1. root@OpenMediaVault:/usr/sbin# omv-snapraid-diff
      2. /usr/sbin/omv-snapraid-diff: line 365: [: -eq: unary operator expected

      Same error from cron window and command line

      Line 365-366


      Source Code

      1. if [ $(omv_config_get "/config/services/snapraid/prehash") -eq 1 ]; then
      2. prehash=" --prehash"
      Edit: If i disable the pre-hash setting it works fine
    • gderf wrote:

      Will you backport this change to the OMV 2.x plugin?
      I wasn't planning on it.

      tx.canyon wrote:

      Still having an error here on my fresh install (OMV 3.x and Snapraid 3.6.4 plugin
      I fixed that shortly after I released 3.6.4. Look up a few posts and you will see 3.6.5 was released.
      omv 4.1.12 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Quick questions before I plan and use SnapRAID:

      1. Does SnapRAID see HDDs merged using mergefs as one big HDD or separate HDDs?
      2. To add the content files to these HDDs, if the HDD is considered full by mergefs, is it still possible to make the content file there even though it has 16GB of space left (I have set mergefs such that it will switch to another HDD when current HDD has 16GB of space left)?
      3. How long do syncing or scrubbing take? Is there a way to calculate rough estimates?

      Thanks!
    • darkarn wrote:

      Does SnapRAID see HDDs merged using mergefs as one big HDD or separate HDDs?
      Both. When setting up snapraid, pretend you don't have mergerfs installed and don't use the mergerfs anywhere in snapraid.

      darkarn wrote:

      To add the content files to these HDDs, if the HDD is considered full by mergefs, is it still possible to make the content file there even though it has 16GB of space left (I have set mergefs such that it will switch to another HDD when current HDD has 16GB of space left)?
      Maybe.

      darkarn wrote:

      How long do syncing or scrubbing take? Is there a way to calculate rough estimates?
      Hard to say. Depends on cpu speed, file size, disk speed, etc.
      omv 4.1.12 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      darkarn wrote:

      Does SnapRAID see HDDs merged using mergefs as one big HDD or separate HDDs?
      Both. When setting up snapraid, pretend you don't have mergerfs installed and don't use the mergerfs anywhere in snapraid.

      darkarn wrote:

      To add the content files to these HDDs, if the HDD is considered full by mergefs, is it still possible to make the content file there even though it has 16GB of space left (I have set mergefs such that it will switch to another HDD when current HDD has 16GB of space left)?
      Maybe.

      darkarn wrote:

      How long do syncing or scrubbing take? Is there a way to calculate rough estimates?
      Hard to say. Depends on cpu speed, file size, disk speed, etc.

      Thanks! I hope SnapRAID will play nice with rsnapshot too (which on principle alone should be fine since SnapRAID also see the hardlinked files as normal files too?)

      Time to get the new HDDs soon too
    • Ankh wrote:

      Hi,

      Got some strange issues with a fresh install.
      When adding drives, the pulldownmenu is empty. I wiped the hard drives but it doesn't matter.
      Any idea's?
      do you mean in SnapRaid tab?

      SnapRaid does not work with raw drives. it is kind of a "pseudo raid" and need an FS on the disk and disk mounted in the system.
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • you looking at it all wrong.

      SnapRaid sits on top of FS. you can use any FS you want on the drives and just put them into SnapRaid setup. it is a independent software that, to some degree, emulate raid functionality.

      same thing with MergerFS. it is not a file system per see, it is a software layer that aggregate a bunch of disks into a single presentable volume but works on top of existing FS. like an intermediary between them and user shares.

      what is great about both a snapraid and mergers is that they work with existing disks with existing FS on them, they are non-destructive to the data, as in you can add drives with data to both on the fly and data will be there for you to use, and you can remove disassemble the pools and data still be there.


      you use SnapRaid for data protection and you use MergerFS for have a single pool to work with using multiple hardware disks underneath.

      unlike ,what I usually call a real raid, hardware or software, which does 2 things at the same time,
      1. pools a bunch of disks into single volume and protects the data on that pool
      SR and MergerFS do only one thing each.
      SnapRaid protects your data from destruction and keeps you running if a hardware drive fails
      and MergerFS allows you to merge (pool) a bunch of drives into a single volume.
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T