Rsync how to reduce log verbosity

1. offizieller Beitrag

    Hi all! Need a little help here. Not really a big problem, but still...
    The verbosity of Rsync is to high for my needs. Any way to reduce it? I just need stats not the (huge) list of files.
    By looking at /var/lib/openmediavault/cron.d/rsync-*, it seems the --verbose option is automatically added by OMV. I could,of course, remove it from the script, but that's volatile.
    Another option is to add the --quiet option to the "extra options" on OMV-GUI, but that would colide with the --verbose already there.
    Worst case scenario would be to create my own script and cron it, but I really like the Rsync module on OMV-GUI.
    So, I guess my real question is, "How to get rid of the --verbose option automatically added to the rsync scripts?"


    Best regards,
    CAM

    Thank you for the quick reply and for confirming the "quiet" option.
    This is what I've got... (like you said, quiet is "on" right?)

    And this is what I see on the logs. Over and over (+679k files)......
    2017/12/27 12:04:03 [30256] .d..t...... /Thunderbird/Profiles/05axvupi.default/calendar-data/
    2017/12/27 12:04:03 [30256] >f.st...... /Thunderbird/Profiles/05axvupi.default/calendar-data/local.sqlite
    2017/12/27 12:04:03 [30256] .d..t...... /Thunderbird/Profiles/05axvupi.default/crashes/
    2017/12/27 12:04:03 [30256] >f..t...... /Thunderbird/Profiles/05axvupi.default/crashes/store.json.mozlz4
    2017/12/27 12:04:03 [30256] .d..t...... /Thunderbird/Profiles/05axvupi.default/datareporting/
    2017/12/27 12:04:03 [30256] >f+++++++++ /Thunderbird/Profiles/05axvupi.default/datareporting/aborted-session-ping
    2017/12/27 12:04:03 [30256] >f..t...... /Thunderbird/Profiles/05axvupi.default/datareporting/session-state.json
    2017/12/27 12:04:03 [30256] >f.st...... /Thunderbird/Profiles/05axvupi.default/startupCache/startupCache.4.little
    2017/12/27 12:04:04 [30256] sent 3,851,027 bytes received 197,495,771 bytes 830,296.07 bytes/sec
    2017/12/27 12:04:04 [30256] total size is 199,686,712,098 speedup is 991.76


    I'll do some more testing (change options, reboot OMV etc). This is in production so, not readily available. I'll post back when I find what I'm doing wrong....


    Best regards
    CAM

    Can't get it to work.
    The "quiet" option doesn't work for me.
    However, if I run the rsync job manually, the "Execute rsync job" window that pops up behaves correctly:
    with quiet on, it just says "Done ..."
    with quiet off, it lists all files as they are rsync'd
    either way /var/log/rsync.log always gets everything
    The --stats option (tanks @macon) is what I want to implement and if I add it, I do get the statistics PLUS the files list (no matter if "quiet" is on or off).
    One more thing:
    the /var/lib/openmediavault/cron.d/rsync-* is updated when I change something; for example, the "quiet" option or the "stats" option appear and disappear from the script if I add or remove them. But the ---verbose option is always there.
    I've tried this on 3.0.91 (production) and 4.0.15-1(Oracle VM testbox). Same results.


    weird....
    CAM

    • Offizieller Beitrag

    Not that much control @flmaxey. Omv doesn’t rewrite the configurations on reboot. Is only on demand, this happens in 99% of the cases when the yellow apply button appears. It also depends if the section or plugin marks more modules to be reconfigured.


    Say for example I create a foobar plugin when I press the save button Omv firstvwrites to the db and marks dirty the current foobar module. When press apply mkconf will rewrite the specific bash configuration filenfor foobar. The foobar mkconf gathers info from the db to generarte this new configuration. If is a service there should be a restart involved.


    In this module section I can add to my code many modules as I can. So I can set it for every write of foobar to the db, mark dirty samba, ftp and scheduled tasks. That can force the rewrite of all those services again.


    You can do it the other way around foobar can listen to changes in other sections. Privileges is an example of this as samba, ftp and netatalk all listen for changes in the sharedfolder db.


    Back to the rsync, the verbose option is hardcoded, there is also no env variable available to suppress it when mkconf acts. Recommendation copy the rsync line to scheduled tasks and take out the verbose option.


    The whole rsync script is located at /var/lib/openmediavault/cron.d/ with rsync prefix.

    Ah.
    Crap.
    So, either live with a HUGE rsync.log (or frequent logrotates) or loose the great rsync OMV interface and use "scheduled jobs"...
    Any chance to modify this "odd behaviour" any time in a near future? Pretty please? :D
    Rsync is a great tool and it's integration in OMV is, IMHO, a very important feature. But the logging does need some revamping. I have some ideas...


    I'm kinda surprised this hasn't been brought up before ?(
    Best regards
    CAM

    @flmaxey - I wasn't in any way trying to say that my requests are more important than anyone elses; in that respect, I think that my post couldn't be more humble (humbler? I need to improve my English...). I do understand that prioritizing issues as a function of available resources is a key to success.
    Having said this, I do believe that rsync is a key tool in the OMV ecosystem and, as such, I feel I'm actually helping the community by reporting issues and actually proposing solutions.


    @all - to have this kind of quick, and detailed response is not common even (specially?) in commercial environments (yes DELL, I'm looking at you!). Thank you.


    Best Regards
    CAM

    @flmaxey Thank you for your kind words :thumbup:
    I try (sometimes VERY hard) not to get into the "old-school" discussion. The real question should be "does our user base happily use it?" or, alternatively "are we prepared to push a different solution" (aka Microsoft Metro Desktop...).
    Snapshots and rsync are not necessarily exclusive and for the kind of user size *I think* OMV aims to, snapshots tend to be (IMO) overkill (I lack a better word; again, English limitations).
    Rsync is robust, reliable, well documented, portable, simple to use, versatile (I could go on, but won't).
    In fact, from my "surroundings" what I see is that small to medium institutions (up to a few 100 machines) like schools, universities, even hospitals use some form of rsync front-end.


    ah, but I digress.
    Thank you all for your time and patience I'll go bother someone else at the ZFS section.
    Can I close this thread?


    Best regards
    CAM

    FYI and for future reference the solution (as always, RTFM was necessary) is to --log-file-format='' (NOT double quotes but 2 single quotes).This defines an empty string as the output format instead of the default %i %n%L. The log-file-format has many options (again, RTFM). However, for this option to work (and I did miss that "tiny" detail for quite some time), the log has to be "redirected" to a file. So, it must be something like --log-file-format='' --log-file=/home/username/foo.log. And since I wanted stats, I can now add --stats.
    Adding this three options to the "extra options" field did the trick. No need to fork and submit a pull request, only a little reading.
    Best regards and, again, thanks


    CAM

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!

Benutzerkonto erstellen Anmelden