Snapraid diff - wrong threshold condition?

  • Hi,


    This is my first post, so hello to everyone.


    I have some strange problem with latest omv-snapraid-diff. There was a thread with "else" error, but it was updated already. Now, tonight strange thing happened:


    First of all - scheduled diff completed and stopped - please note the start and stop times :


    Now, second mail - this is something new:


    So one second AFTER diff, there goes SYNC script, that says 1165 is BELOW 100 and start SYNCINC. I'm no script master, but trying to view current latest omv-snapraid-diff, it seems to me, there is no exit condition after DELETE threshold check. Can someone please confirm, if it's a bug, or there is something wrong with my installation, and I shall configure it from sractch?
    I have NO separate cron job for sync job - looks like its called from diff.
    All the threshold stuff was working till last two updates, but it's strange no one is complaining about it :/

  • I'm seeing this exact problem as well. Update threshold disabled (0), delete threshold enabled (50). Deleted files: 137, so it shouldn't run a SYNC, but it does anyways. I've disabled the script and am running manual diff/sync for now. Hopefully this gets fixed soon.

  • I've noticed the same issue. I haven't dug in too much, but my first guess is maybe it has something to do with the update threshold? I've been using snapraid-diff for years, and never had an issue until recently (within a month), and have noticed that an update threshold (which I have disabled) is now included in my daily sync emails. The behavior is the same as described above. It seems like the script is running twice. The first email I get gives a warning that deleted files is above the threshold, then a second email (just after) comes in saying that sync is complete.


    Example of first email where it works correctly to stop the sync:
    201615 equal
    87 added
    340 removed
    328 updated
    0 moved
    0 copied
    0 restored
    There are differences!
    SnapRAID DIFF finished - Sun Sep 8 23:30:33 EDT 2019
    ----------------------------------------
    Changes detected [A-87,D-340,M-0,C-0,U-328] -> there are updated files (328) but update threshold (0) is disabled. Running SYNC Command
    Number of deleted files (340) exceeded threshold (20). NOT proceeding with sync job.
    Please run sync manually if this is not an error condition


    Yours, SnapRAID-diff script



    Then right after the sync proceeds:
    Changes detected [A-87,D-340,M-0,C-0,U-328] -> deleted files (340) is below threshold (20). Running SYNC Command
    SnapRAID SYNC Job started - Sun Sep 8 23:30:34 EDT 2019

  • ryecoaaron, I'd agree. I'm pretty novice at coding, and this type of script change and edits are way beyond my abilities. However, looking at the link you sent, the options do look cleaner, with a lot of added abilities to proceed with or stop the sync.


    I looked at the current snapraid-diff script some more, and it looks like the sync command (line 446) happens regardless of update or delete thresholds (there's no if block for the sync command except if there are changes). Note, I didn't copy that code here due to the message length. So from what I can tell from current script, if the update threshold is exceeded, an email will be sent, but it will continue with the other checks and continue to sync. If the delete threshold is exceeded, similarly an email will send, but the sync will continue. This also explains why we're getting multiple emails from the same sync call. There's no "exit" if the thresholds are exceeded.


    I've made a new file snapraid-diff.fix and I manually commented out the update threshold (since I don't use it) and added an else for the sync and scrub commands - so they won't execute if the deleted files is above the threshold. Note this isn't very clean, but it's a quick fix that works for me. Here's line 385 and down as an example if this helps others. Again, probably not a good long term fix, but more of a makeshift that seems to work for me in a few quick tests:



  • I use the script that can be found here and have no problems with it.

    I could remove the diff script and diff script parameters from the plugin if people used that. So, I like that idea :)

    omv 5.5.17-3 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.4.2
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I use the script that can be found here and have no problems with it.


    https://gist.github.com/mtompk…be36064c237da3f39ff5cc49d


    There are others that can be found by searching.

    Can you use it as a drop in - i.e. without installing anything else and just modifying a few parameters? Looking at the script, it says it requires mailx and python markdown. I guess my concern is having to monkey with something else, install a few more programs, etc. The nice thing about the snapraid plugin with OMV is you just push a button to install, configure with the webui, and done. Very simple, and a few buttons to push.


    Personally, I realize it means maintenance, but I like the parameters in the plugin. I'd imagine most beginners would too, vs trying to play with a different script and make sure it works for their needs.


    I'll dig in more into the mtompkins script and the existing v 1.1 omv-diff script and see if I can combine some new functions into it. May take a little while as I'm still learning / pretty newb at bash coding.

  • I don't recall having to install anything extra to use that script, whatever it requires must already be installed in my use case.


    As I said there are other scripts out there. I used to use this one:



    Code
    https://gist.github.com/bfg100k/87a1bbccf4f15d963ff7

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

  • All - I know there's other scripts for SnapRaid, but I didn't spend too much time looking in to those as it seemed like more of starting over, vs just having the plugin that worked as it always had. Anyway, I created a fork, where I corrected the omv-snapraid-diff script to correctly stop the sync if the deleted or update threshold is met or exceeded. I've been using this updated script for almost a month with zero issues. This update / correction can be merged with the main branch, but we need some more testers per discussion with ryecoaaron.


    To those that are still using the plugin script and having this issue of sync continuing despite thresholds being met, you can download the omv-snapraid-diff here: https://github.com/danmcg13/op…raid/tree/master/usr/sbin. If you do use it please report back with any issues you have or if it's working great reply back with that too so we can get more testing in. Hope this helps!


    Note, this doesn't add any new features or improve the script, but just corrects the issue of the script telling snapraid to sync regardless of updated or deleted file counts, as it does today. Thanks.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!