persistent "monit alert -- Resource limit matched localhost" emails started last few days

    • OMV 2.x
    • persistent "monit alert -- Resource limit matched localhost" emails started last few days

      hello.
      I have been running OMV for the last several weeks.

      for the last several days I have been getting tons of email alerts with "t "monit alert -- Resource limit matched localhost" emails started last few days"
      Description: cpu system usage of 95.3% matches resource limit [cpu system usage>95.0%]
      and
      Description: 'localhost' cpu system usage check succeeded [current cpu system usage=63.0%]

      in process info I see a process from root taking up 145+ % of CPU command "echo find"
      this is the only process I can see with this kind of CPU time.

      server is running ok, except for the email I can not see anything wrong with it.

      Source Code

      1. ================================================================================
      2. = OS/Debian information
      3. ================================================================================
      4. Distributor ID: debian
      5. Description: Debian GNU/Linux 7 (wheezy)
      6. Release: 7.11
      7. Codename: wheezy
      8. ================================================================================
      9. = openmediavault information
      10. ================================================================================
      11. Release: 2.2.5
      12. Codename: Stone burner
      13. ================================================================================
      14. ================================================================================
      15. = Locale
      16. ================================================================================
      17. en_US.UTF-8
      18. ================================================================================
      19. = System information
      20. ================================================================================
      21. Linux atlas 3.16.0-0.bpo.4-amd64 #1 SMP Debian 3.16.7-ckt25-2~bpo70+1 (2016-04-12) x86_64 GNU/Linux
      22. ================================================================================
      23. = Uptime
      24. ================================================================================
      25. 09:14:45 up 11 days, 19:54, 0 users, load average: 4.25, 5.86, 6.62
      Display All
      Files
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • root@atlas:/# ps aux | grep "echo"
      root 8389 0.0 0.0 7848 1908 pts/2 S+ 11:48 0:00 grep echo
      root 11958 154 0.0 74044 5880 ? Ssl Jun19 3397:05 echo "find"
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • ok, but what is it ?

      also my SSH service keeps going off, I have never seen things like this.
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T

      The post was edited 1 time, last by vl1969 ().

    • I don't know. Some weird command stuck in a loop.

      Post the output of: ps aux
      omv 4.1.19 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • no can do, my connection just went out.
      will try later when get home.
      thanks
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • ok, I have killed the process before as ryecoaaron sugested
      but almost immediately a new process with new id started with command "who"


      Source Code

      1. root 22232 48.1 0.0 74472 7740 ? Ssl Jun22 690:45 who


      the process list on OMV UI identifies the process as "bxigzbtgri"
      and it is the same one that was causing issue last time just different commnad.

      Source Code

      1. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
      2. 22232 root 20 0 74472 7740 412 S 101 0.0 691:35.18 bxigzbtgri



      anyone know what this is?
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • You have been hacked! Easy root password maybe?? Disconnect that system from internet immediately. Personally, I would reinstall.
      omv 4.1.19 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • yeah I figured out this already when I try to look up the file on my PC.
      it screamed Malware and removed the temp folder files :)

      I killed the process and removed the file from "/usr/bin"
      I changed password for admin and root as well.

      so far I do not see any unusual activity.
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • Glad you trust that nothing else is on it. I wouldn't. I would at least check it with rkhunter.
      omv 4.1.19 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • I don't trust it , but at the moment I can not rebuild it.
      I will scan it with rkhunter though thanks .


      [EDIT]
      OK I have run "rkhunter" (log attached)
      except for some outdated apps all looks ok.
      Files
      • rkhunter.txt

        (117.74 kB, downloaded 157 times, last: )
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T

      The post was edited 1 time, last by vl1969 ().

    • ok, I will have to rebuild.
      I got a strange process again out of nowhere.
      have to wait couple of weeks though but I have shut down the server for now.

      is there a way for me to preserve the config for SnapRaid and mergerfs so I do not have to redo it again?
      omv 3.0.56 erasmus | 64 bit | 4.7 backport kernel
      SM-SC846(24 bay)| H8DME-2 |2x AMD Opteron Hex Core 2431 @ 2.4Ghz |49GB RAM
      PSU: Silencer 760 Watt ATX Power Supply
      IPMI |3xSAT2-MV8 PCI-X |4 NIC : 2x Realteck + 1 Intel Pro Dual port PCI-e card
      OS on 2×120 SSD in RAID-1 |
      DATA: 3x3T| 4x2T | 2x1T
    • Not really. Those two plugins (especially mergerfs) are easy to configure. I would just cut&paste the snapraid config for later setup. After you setup the new system, compare the config to the old one. If they are the same, you should be good to go. Worst case, you would just have to do a full sync again. neither plugin will cause file loss.
      omv 4.1.19 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!