What causes Monit Alerts

    • What causes Monit Alerts

      Hello all,

      this week I received 20 Emails with Monit Alerts one morning between 6:06 am and 7:30 am.
      All the notification emails either stated "Recource limit matched Service localhost (loadavg(5min) of 2.6 matches resource limit [loadavg(5min)>2.0]) or "Resource limit suceeded Service localhost ('localhost' loadavg(1min) check succeeded [current loadavg(1min)=4.0]).

      Now I'm wondering what caused those alerts as there where no activities initialised by me during that period (like backups or whatever). Does anyone have a clue what might be responsible for the alerts or does anybody know a tool to trace the origin for such a bunch of notofications?

      Thanks in advance.

      McFly3025.
    • Could be collectd running amok again.

      Greetings
      David
      "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"

      Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.


      Upload Logfile via WebGUI/CLI
      #openmediavault on freenode IRC | German & English | GMT+1
      Absolutely no Support via PM!

      I host parts of the omv-extras.org Repository, the OpenMediaVault Live Demo and the pre-built PXE Images. If you want you can take part and help covering the costs by having a look at my profile page.
    • Hi there,
      I observe more or less similar mail notifications since I dared to upgrade form Sardaukar to Kralizec 1.9 last week. I usually receive a pair of monit notification within one minute:

      1. notification: monit alert -- Resource limit matched localhost: cpu wait usage of 100.0% matches resource limit [cpu wait usage>95.0%]
      2. notification: monit alert -- Resource limit succeeded localhost: 'localhost' cpu wait usage check succeeded [current cpu wait usage=1.5%]


      What makes me wonder:
      A. On Sardaukar I did not receive such messages
      B. Messages are sent sometimes when the OMV server starts up
      C Messages are sent sometimes when the OMV server resumes from hibernation (via AutoShutDown plugin)
      I could not find a pattern up to now, except that the messages were only sent on start up or resuming conditions.

      When I watch the processes on my OMV server with "top" I cant find any process which may be the cause for such a load. So, as the original poster wrote, I also do not have a clue what is causing thins load not do I have an idea how to tackle the issue. Perhaps someone has a hint?

      Thanks in advance,
      topi
      OpenMediaVault 3.x (most recent stable version) -- 64 bit -- OMV-Extras 3.x (most recent stable version) -- Default Kernel
    • For me its collectd that sometims clogs up the CPU really heavy. Wouldn't wonder me if it does this too for you guys when you resume from hibernation.

      Greetings
      David
      "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"

      Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.


      Upload Logfile via WebGUI/CLI
      #openmediavault on freenode IRC | German & English | GMT+1
      Absolutely no Support via PM!

      I host parts of the omv-extras.org Repository, the OpenMediaVault Live Demo and the pre-built PXE Images. If you want you can take part and help covering the costs by having a look at my profile page.
    • You may check if this always happen at the same time if you are running 24/7. I assume that it is cron-apt that is checking for new updates. You may have a look into /var/log/cron-apt/log to check when the last execution was. Maybe this correlates with your emails.
      Absolutely no support through PM!

      I must not fear.
      Fear is the mind-killer.
      Fear is the little-death that brings total obliteration.
      I will face my fear.
      I will permit it to pass over me and through me.
      And when it has gone past I will turn the inner eye to see its path.
      Where the fear has gone there will be nothing.
      Only I will remain.

      Litany against fear by Bene Gesserit
    • Thank you for your replies.

      Actually my system has been up for almost 40 days without any interruptions. I recieived a couple of notifications during that time, but it was only last Wednesday that I received that amount for appr. 90 minutes.
      I was checking that apt-logfile and there was nothing suspicious catching my eyes. So maybe it was really this collectd.

      Best regards.

      McFly
    • Hi votdev,

      I checked the situation now for a few days when the mail notifications appear. I can say for sure that they appear only when my system starts up or when it resumes from hibernation. It shows alwas the same pattern, I usually receive a pair of monit notifications within one minute:

      Source Code

      1. 1. notification: monit alert -- Resource limit matched localhost: cpu wait usage of 100.0% matches resource limit [cpu wait usage>95.0%]
      2. 2. notification: monit alert -- Resource limit succeeded localhost: 'localhost' cpu wait usage check succeeded [current cpu wait usage=1.5%]


      So, would there be a way to make monit somehow more resilient with this cpu usage peaks?

      Thanks,
      topi
      OpenMediaVault 3.x (most recent stable version) -- 64 bit -- OMV-Extras 3.x (most recent stable version) -- Default Kernel
    • You can customize the settings. Have a look into /usr/share/openmediavault/mkconf/monit.d/system. Set the environment variables to your prefered valies in /etc/default/openmediavault. To apply the new values you have to execute

      Source Code

      1. # omv-mkconf monit
      Absolutely no support through PM!

      I must not fear.
      Fear is the mind-killer.
      Fear is the little-death that brings total obliteration.
      I will face my fear.
      I will permit it to pass over me and through me.
      And when it has gone past I will turn the inner eye to see its path.
      Where the fear has gone there will be nothing.
      Only I will remain.

      Litany against fear by Bene Gesserit
    • Please check the monit manual pages to get the required information. I do not know what you want to achieve, so you have to do it on your own. The default OMV values should fit on most systems.

      Manual pages: mmonit.com/monit/documentation/monit.html
      Absolutely no support through PM!

      I must not fear.
      Fear is the mind-killer.
      Fear is the little-death that brings total obliteration.
      I will face my fear.
      I will permit it to pass over me and through me.
      And when it has gone past I will turn the inner eye to see its path.
      Where the fear has gone there will be nothing.
      Only I will remain.

      Litany against fear by Bene Gesserit
    • I'd like to modify the MAIL-FORMAT statement. But seems that the changes I make get overwritten by omv-mkconf monit. So where can I define MAIL-FORMAT that omv-mkconf monit will use? Primarily, I'd like to change the subject line. (I'm still fighting the distinction of Resource limit matched vs. Resource limit succeeded).

      You mention in another post that I can modify:
      OMV_MONIT_SERVICE_SYSTEM_LOADAVG_1MIN_MULT
      OMV_MONIT_SERVICE_SYSTEM_LOADAVG_5MIN_MULT

      but I'm not sure how or what I may modify it to be.

      I event tried creating my own injection as /etc/monit/conf.d/openmediavualt-mailformat.conf, but it get's wiped out with omv-mkconf monit

      The post was edited 1 time, last by bkeadle ().

    • This happen to me too. My server has more than enough power & ram to run OMV.

      At first I thought the ClamAV, antivirus, using too much resource.
      But after I uninstalled it, I still get notification about "Resource limit succeeded Service localhost".
      It happened every time the server is wake up/resumed from hibernation or shutdown.
      Can't find the solution.

      OMV v2.1.1
      OMV v4.0
      Asus Z97-A/3.1; i3-4370
      32GB RAM Corsair Vengeance Pro
      4x3TB RAID10