[HowTo] SnapRAID in OMV

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • bmhj2 wrote:

      jollyrogr wrote:

      bmhj2 wrote:

      Thanks. From my reading of the manual, -e only applies to the 'fix' command.

      Should I use -E (upper case), or am I misunderstanding the manual?

      Sorry - just want to be clear before setting this running.

      Thanks.
      You're right, my mistake. Should be snapraid -E sync
      Great. Thanks for your help.
      Did that process work for you? I'm getting the same error, and I follwed the process listed here. Deleted drive from OMV gui, went to terminal and ran command snapraid -E sync, and got this error:

      Source Code

      1. Disk 'd2' with uuid 'abee6d94-71ad-42ec-ab28-bc921527b6a8' not present in the configuration file!
      2. If you have removed it from the configuration file, please restore it
      How did you resolve your error? Thanks.

      The post was edited 1 time, last by kmal808 ().

    • kmal808 wrote:

      Did that process work for you? I'm getting the same error, and I follwed the process listed here. Deleted drive from OMV gui, went to terminal and ran command snapraid -E sync, and got this error:

      Disk 'd2' with uuid 'abee6d94-71ad-42ec-ab28-bc921527b6a8' not present in the configuration file!
      If you have removed it from the configuration file, please restore it How did you resolve your error? Thanks.
      Hello kmal. No, that didn't do the job for me, I got the same error as you quote here (different disk ID, obvs) and as a result I have not yet been able to remove the SSD from my Snapraid array. I haven't followed up or tried anything else yet, as my priorities were elsewhere, but I'm glad you reminded me: if @jollyrogr or anyone else can point me in the right direction here I would be grateful.

      Thanks!
    • Not sure that it matters, but looking at the snapraid manual, I see that they list the command as
      snapraid sync -E

      Try running with the -E at the end instead of the way I had it before?

      They also list the steps to remove the disk as follows:

      Source Code

      1. To remove a data disk from the array do:
      2. •Change in the configuration file the related "disk" option to point to an empty directory
      3. •Remove from the configuration file any "content" option pointing to such disk
      4. •Run a "sync" command with the "-E, --force-empty" option: snapraid sync -E
      5. The "-E" option tells at SnapRAID to proceed even when detecting an empty disk.
      6. •When the "sync" command terminates, remove the "disk" option from the configuration file.
      7. Your array is now without any reference to the removed disk.
      Not sure how that's supposed to work with OMV since I don't think you can use the GUI to edit the configuration file to point to an empty directory. May need to try logging into the shell, and (as root) do the steps listed.
    • I think @bmhj2 has already manually edited the config file and had the same results/error. I read the instructions from the SnapRAID guide that you referenced, but wasn’t sure about editing the config file manually, so I searched the forum and came across this thread. My real problem is that I can’t get the first sync to finish. I’ve tried just about everything I can find or think of and still get the “Warning! The array is not fully synced. You have a sync in progress at 99%” when checking Snapraid status. I’ve read that docker files and Plex data don’t mix well with Snapraid, so that’s why I’m trying to remove a drive(the drive that contains these files).
    • So you can't even get the initial sync to complete? How long did you let it go? It can take many hours to complete. I don't use docker or plex and have had no issues. Snapraid is designed to be used in situations where files change very infrequently. Perhaps docker and plex are constantly changing files and causing issues that way. If there are certain affected directories, those can be easily excluded from the snapraid array. If plex or docker is mucking with all your media, then it probably just wont work.
      • The initial sync ran and the log shows it as finishing, although while syncing it did throw errors for modifying files during sync, and those files were from docker. When I check Snapraid status is where I get the error about “sync in progress”. That combined with the errors during the sync leads me to believe docker is modifying files during the sync, causing the inability to completely sync. To remedy this I tried to remove the disk that the docker files live on from the array, but am not able to remove the disk, which lead me here. I think if I can remove this disk from the array, the sync should run w/o error.
    • @jollyrogr yeah thanks for that tip. I would rather remove the drive and then keep that drive out of both Snapraid & mergerfs, but this will work for now. Wish I would have known what I know now prior to installing the plugin. Ha. On that note, would uninstalling and reinstalling/reconfiguring the Snapraid plugin create any problems? What happens to the data on the parity drive and content files when the plugin is uninstalled? May have to look into that as well if I can’t get removing a drive to work. @gderf thanks for the info, I’ll keep that drive out of the pool.
    • Thanks to others for further discussion here.

      kmal808 wrote:

      I think @bmhj2 has already manually edited the config file and had the same results/error. I read the instructions from the SnapRAID guide that you referenced, but wasn’t sure about editing the config file manually, so I searched the forum and came across this thread. My real problem is that I can’t get the first sync to finish. I’ve tried just about everything I can find or think of and still get the “Warning! The array is not fully synced. You have a sync in progress at 99%” when checking Snapraid status.

      Yes, that's right: I have tried editing the config file via CLI and via the plugin, with no success using either method.

      It seems as if @kmal808 have been encountering the same error, caused by files which changed somehow without that being picked up by the parity. In my case these errors have only related to two drives:
      1. The SSD shown in my earlier post, which I wanted to remove because it contains Plex databases and Docker config data, and I realised (too late) that Snapraid is not the best way to back those up.
      2. An HDD not mentioned before, which has developed problems since I last posted. This is almost certainly because I used Beets to re-tag the metadata on a large music collection, meaning that many of the files now show errors on Snapraid, presumably because of some snafus created by that Beets import taking place while I was also messing around with the "snapraid sync -F" command trying to remove the SSD from the array (I have also, as indicated in my original post, since added a second HDD parity drive).
      Stating the problem very simply, having first misunderstood how Snapraid works and created a setup which was likely to have problems, I have then compounded my error by making lots of file changes in quick succession so that I am left with parity files which are incorrect. As a consequence I can't run a successful sync, because Snapraid is returning false positives - suggesting there are data errors where I am confident there aren't.

      For (1) I have managed to stop the error messages by adding exclusions for the Docker and Plex folders on the SSD as described by @kmal808 above. If I understand things correctly, this has removed files on the SSD from the snapraid.parity and the snapraid.content files on a subsequent sync, and this seems to have acted as the bandaid that @kmal808 describes above. The outstanding question is still this: how to remove the drive from the array completely?

      For (2) I have not yet done anything, but I am wondering whether the following will work:
      • add temporary exclusions for the HDD to the config
      • run a sync (on the assumption that this will then remove the faulty parity data)
      • remove the exclusions
      • run another sync (on the assumption that this will force a recalculation of the parity)
      Or...

      it does strike me that another less fiddly solution would be (as @kmal808 seems to suggest, and @gderf advises against) to just nuke the array and start from scratch. @kmal808 suggests uninstalling the plugin, but doesn't know how this would affect the snapraid.parity and snapraid.content files. I assume it would be possible to exercise this option by simply deleting all the parity and content files and running a new sync, but I have already proven myself to be dumber than I thought here so would prefer to . I'd be willing to take the risk with the data, since if necessary I can restore from a backup - but I suppose I would ask the following questions:
      1. Can I remove the SSD from the array and force a complete rebuild of the HDD parity data without exercising my nuclear option?
      2. How complicated will this be?
      3. If I exercise the nuclear option, what is the simplest means of doing so?
      I should finally add: forgive me, fellow forum users, for I have sinned! Hope you don't mind helping me dig myself out of this hole.
    • Hi everyone. I can't seem to find the ability to search this thread, so I hope this isn't already answered. I have snapraid setup to send an e-mail to me from its nightly sync/scrub per the diff script.

      Everything is working fine except the e-mail is a bit spammy. It gives me every change to the % and time remaining. Is it possible to exclude these redundant lines from the e-mail?

      example below -
      Using 4527 MiB of memory for the FileSystem.
      Initializing...
      Scrubbing...
      Using 168 MiB of memory for 32 blocks of IO cache.
      0%, 2 MB
      0%, 348 MB
      0%, 1487 MB
      0%, 2710 MB
      0%, 3909 MB
      0%, 5143 MB, 1198 MB/s, CPU 72%, 1:53 ETA
      0%, 6331 MB, 1196 MB/s, CPU 72%, 1:53 ETA
      0%, 7568 MB, 1203 MB/s, CPU 72%, 1:53 ETA
      0%, 8767 MB, 1202 MB/s, CPU 72%, 1:53 ETA
      0%, 9975 MB, 1203 MB/s, CPU 72%, 1:53 ETA
      0%, 11201 MB, 1214 MB/s, CPU 71%, 1:52 ETA
      0%, 12415 MB, 1214 MB/s, CPU 71%, 1:52 ETA
      0%, 13614 MB, 1212 MB/s, CPU 71%, 1:52 ETA
      0%, 14846 MB, 1214 MB/s, CPU 71%, 1:52 ETA
      0%, 16045 MB, 1213 MB/s, CPU 71%, 1:52 ETA
      0%, 17262 MB, 1212 MB/s, CPU 71%, 1:52 ETA
      0%, 18476 MB, 1212 MB/s, CPU 70%, 1:52 ETA
      0%, 19704 MB, 1213 MB/s, CPU 70%, 1:52 ETA
      0%, 20880 MB, 1211 MB/s, CPU 70%, 1:52 ETA
      .... all the way to 100%

      is there a way to have this removed from the nightly e-mail?
    • You would need to edit the diff script. Line 422 sends all output of the sync command to a variable and elsewhere in the script that output is included in the email.

      Source Code

      1. $SNAPRAID_BIN sync${prehash} >> $TMP_OUTPUT

      Line 444 includes all the scrub output.

      Source Code

      1. $SNAPRAID_BIN scrub -p $SCRUB_PERCENT -o $SCRUB_OLDER_THAN_DAYS >> $TMP_OUTPUT