Fresh OMV 7 installation --> rrdcached and nginx alerts

bennySB · Dec 12th 2024

Hi everyone,

Yesterday I completely reinstalled OMV because I suddenly got error messages after a restart (the same ones I'm about to post) and also did a CPU swap.

Now I thought I was rid of the errors afterwards, but they are still there. However, I receive error messages by e-mail, but the services are running normally.

Could it be that OMV just takes too long to start? After a restart, the SSH connection is re-established very quickly, but the WebUI takes quite a long time to start, and as soon as it is accessible, the messages start coming in.

I receive the following error messages by e-mail:

Code

Service:     rrdcached
Event:       Does not exist
Description: process is not running

This triggered the monitoring system to: restart

Code

Host:        NAS
Date:        Thu, 12 Dec 2024 20:27:23
Service:     nginx
Event:       Does not exist
Description: process is not running

This triggered the monitoring system to: restart

Code

Host:        NAS
Date:        Thu, 12 Dec 2024 20:27:23
Service:     rrdcached
Event:       Execution failed
Description: failed to start (exit status -1) -- Program '/bin/systemctl start rrdcached' timed out after 30 s

This triggered the monitoring system to: alert

Code

Host:        NAS
Date:        Thu, 12 Dec 2024 20:28:22
Service:     rrdcached
Event:       Exists
Description: process is running with pid 1198

This triggered the monitoring system to: alert

Code

Host:        NAS
Date:        Thu, 12 Dec 2024 20:28:22
Service:     rrdcached
Event:       Execution succeeded
Description: process is running after previous exec error (slow starting or manually recovered?)

This triggered the monitoring system to: alert

Code

Host:        NAS
Date:        Thu, 12 Dec 2024 20:28:22
Service:     nginx
Event:       Exists
Description: process is running with pid 1171

This triggered the monitoring system to: alert

I checked the status of the services:

Code

root@NAS:~# systemctl status rrdcached.service
● rrdcached.service - LSB: start or stop rrdcached
     Loaded: loaded (/etc/init.d/rrdcached; generated)
     Active: active (running) since Thu 2024-12-12 20:27:51 CET; 12min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 1162 ExecStart=/etc/init.d/rrdcached start (code=exited, status=0/SUCCESS)
      Tasks: 8 (limit: 18963)
     Memory: 12.5M
        CPU: 420ms
     CGroup: /system.slice/rrdcached.service
             └─1198 /usr/bin/rrdcached -B -F -f 3600 -w 900 -b /var/lib/rrdcached/db/ -j /var/lib/rrdcached/journal/ -p /run/rrdcached.pid -l unix:/run/rrdcached.sock

Dez 12 20:27:51 NAS systemd[1]: Starting rrdcached.service - LSB: start or stop rrdcached...
Dez 12 20:27:51 NAS rrdcached[1162]: rrdcached started.
Dez 12 20:27:51 NAS systemd[1]: Started rrdcached.service - LSB: start or stop rrdcached.

Display More

Code

root@NAS:~# systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-12-12 20:27:51 CET; 13min ago
       Docs: man:nginx(8)
    Process: 1152 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 1170 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 1171 (nginx)
      Tasks: 9 (limit: 18963)
     Memory: 8.0M
        CPU: 20ms
     CGroup: /system.slice/nginx.service
             ├─1171 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;"
             ├─1172 "nginx: worker process"
             ├─1173 "nginx: worker process"
             ├─1174 "nginx: worker process"
             ├─1175 "nginx: worker process"
             ├─1176 "nginx: worker process"
             ├─1177 "nginx: worker process"
             ├─1180 "nginx: worker process"
             └─1182 "nginx: worker process"

Dez 12 20:27:51 NAS systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server...
Dez 12 20:27:51 NAS systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server.

Display More

As you can see, both services work without any problems. Therefore, I simply assume that there is a time problem.

I have also tried to set the delay according to proftpd, rrdcached and nginx slow to start on boot since 7.4.7-1 update?, unfortunately without success.

The system is running on a generic x86-64 hardware (Intel i3-10100 CPU & MSI B460M Pro Mainboard).

Do you have any sugetions for for me?

bennySB · Dec 19th 2024

The error still exists with the current version 7.4.17-1.

Soma · Dec 19th 2024

You can check what is going on with:

sudo systemd-analyze blame

For eg:

Code

USER@HOST:~ $ sudo systemd-analyze blame
9min 46.476s fstrim.service
 1min 8.006s folder2ram_startup.service
     34.102s docker.service
     22.625s smartmontools.service
     21.867s containerd.service
     21.804s postfix@-.service
     21.602s php8.2-fpm.service
     17.775s samba-ad-dc.service
     17.734s srv-dev\x2ddisk\x2dby\x2duuid\x2de2d03e5f\x2d5d48\x2d4fd2\x2dbbce\x2d68e8ebeab41b.mount
     14.775s systemd-udev-settle.service
     13.740s nfs-server.service
     12.749s dev-sdg2.device
     12.564s systemd-journal-flush.service
     10.131s networking.service
      7.178s rrdcached.service
      7.023s chrony.service
      5.739s man-db.service
      5.314s wg-quick@wgnet1.service
      5.211s systemd-modules-load.service
      5.017s systemd-tmpfiles-setup-dev.service
      3.643s systemd-udevd.service
      3.581s srv-dev\x2ddisk\x2dby\x2duuid\x2d541a4acb\x2db6c4\x2d497f\x2db19d\x2d86e478a37773.mount
      2.936s srv-dev\x2ddisk\x2dby\x2duuid\x2d7e674a01\x2dd0b4\x2d40c6\x2da27e\x2d3aa187bc9ad7.mount
      2.921s systemd-networkd-wait-online.service
      1.936s proc-fs-nfsd.mount
      1.917s logrotate.service
      1.822s lvm2-monitor.service
      1.746s systemd-sysusers.service
      1.726s systemd-random-seed.service
      1.709s srv-mergerfs-merged_media.mount
      1.636s systemd-sysctl.service
      1.491s srv-dev\x2ddisk\x2dby\x2duuid\x2d2cc04ef0\x2d6019\x2d4a4a\x2d9bfb\x2dfcdc2e4beb59.mount
      1.148s apt-daily.service
       775ms smartctl-hdparm.service
       725ms apt-daily-upgrade.service
       594ms srv-dev\x2ddisk\x2dby\x2duuid\x2d60fcc509\x2d8d91\x2d4963\x2d8a89\x2d8be1249b9f2d.mount
       464ms openmediavault-beep-up.service
       425ms systemd-networkd.service
       359ms apparmor.service
       337ms phpsessionclean.service
       332ms loadcpufreq.service
       299ms user@1000.service
       283ms systemd-resolved.service
       228ms nginx.service
       223ms nut-driver@ups.service
       220ms srv-dev\x2ddisk\x2dby\x2duuid\x2d425c5c15\x2db46e\x2d4893\x2dbbbe\x2d24b743c25f1f.mount
       215ms ssh.service
       182ms collectd.service
       179ms avahi-daemon.service
       171ms srv-dev\x2ddisk\x2dby\x2duuid\x2d87961ac3\x2d5cc6\x2d47ad\x2da289\x2d0fa0f357b83b.mount
       162ms monit.service
       140ms dbus.service
       131ms modprobe@drm.service
       131ms systemd-logind.service
       120ms systemd-journald.service
       119ms systemd-udev-trigger.service
       114ms nut-driver-enumerator.service
       111ms lm-sensors.service
       110ms nfs-idmapd.service
       110ms systemd-tmpfiles-clean.service
        93ms e2scrub_reap.service
        91ms cpufrequtils.service
        91ms keyboard-setup.service
        79ms systemd-remount-fs.service
        78ms nfsdcld.service
        76ms dpkg-db-backup.service
        67ms boot-efi.mount
        61ms systemd-fsck@dev-disk-by\x2duuid-2cc04ef0\x2d6019\x2d4a4a\x2d9bfb\x2dfcdc2e4beb59.service
        60ms modprobe@dm_mod.service
        58ms systemd-fsck@dev-disk-by\x2duuid-425c5c15\x2db46e\x2d4893\x2dbbbe\x2d24b743c25f1f.service
        52ms export-pote\x2dmedia.mount
        51ms modprobe@efi_pstore.service
        51ms systemd-tmpfiles-setup.service
        46ms console-setup.service
        46ms openmediavault-cleanup-php.service
        45ms systemd-fsck@dev-disk-by\x2duuid-60fcc509\x2d8d91\x2d4963\x2d8a89\x2d8be1249b9f2d.service
        41ms openmediavault-cleanup-monit.service
        40ms rpcbind.service
        39ms dev-hugepages.mount
        38ms systemd-fsck@dev-disk-by\x2duuid-e2d03e5f\x2d5d48\x2d4fd2\x2dbbce\x2d68e8ebeab41b.service
        38ms systemd-update-utmp.service
        37ms dev-mqueue.mount
        37ms ifupdown-pre.service
        35ms sys-kernel-debug.mount
        34ms sys-kernel-tracing.mount
        31ms systemd-binfmt.service
        30ms nfs-mountd.service
        29ms docker.socket
        29ms kmod-static-nodes.service
        28ms openmediavault-engined.service
        27ms wpa_supplicant.service
        26ms modprobe@configfs.service
        26ms rpc-statd.service
        25ms modprobe@loop.service
        25ms user-runtime-dir@1000.service
        21ms modprobe@fuse.service
        19ms systemd-network-generator.service
        17ms e2scrub_all.service
        16ms systemd-fsck@dev-disk-by\x2duuid-541a4acb\x2db6c4\x2d497f\x2db19d\x2d86e478a37773.service
        16ms systemd-fsck@dev-disk-by\x2duuid-7e674a01\x2dd0b4\x2d40c6\x2da27e\x2d3aa187bc9ad7.service
        15ms systemd-fsck@dev-disk-by\x2duuid-87961ac3\x2d5cc6\x2d47ad\x2da289\x2d0fa0f357b83b.service
        14ms run-rpc_pipefs.mount
        10ms proc-sys-fs-binfmt_misc.mount
         8ms rpc-statd-notify.service
         8ms systemd-update-utmp-runlevel.service
         8ms systemd-user-sessions.service
         6ms rsyslog.service
         5ms tmp.mount
         5ms openmediavault-firewall.service
         4ms sys-fs-fuse-connections.mount
         4ms sys-kernel-config.mount
         3ms postfix.service
        49us folder2ram_shutdown.service
        35us blk-availability.service

Display More

bennySB · Dec 19th 2024

Code

root@NAS:~# sudo systemd-analyze blame
2min 244ms systemd-networkd-wait-online.service
   30.499s apt-daily-upgrade.service
    5.992s nut-driver@Eaton.service
    4.695s systemd-udev-settle.service
    4.635s systemd-journald.service
    3.365s docker.service
    2.707s openmediavault-issue.service
    1.303s smartmontools.service
     741ms smartctl-hdparm.service
     679ms quotaon.service
     463ms openmediavault-beep-up.service
     255ms systemd-fsck@dev-disk-by\x2duuid-1d5ab1fe\x2d926c\x2d449b\x2daaef\x2d3668de0848c7.service
     218ms systemd-fsck@dev-disk-by\x2duuid-152040a4\x2de1f5\x2d4944\x2dbde5\x2d0652ea0edaa1.service
     189ms postfix@-.service
     175ms systemd-fsck@dev-disk-by\x2duuid-74485af4\x2d9495\x2d4e4e\x2d8250\x2db560f7e07de1.service
     171ms dev-nvme0n1p2.device
     169ms systemd-fsck@dev-disk-by\x2duuid-0108ae5d\x2d861b\x2d4524\x2d80d7\x2d0d708e6e2d73.service
     168ms loadcpufreq.service
     147ms systemd-fsck@dev-disk-by\x2duuid-32e967a1\x2d07a2\x2d41b0\x2da546\x2df0aaa2764952.service
     146ms systemd-journal-flush.service
     140ms nfs-server.service
     130ms containerd.service
     126ms systemd-networkd.service
     109ms nmbd.service
     107ms lvm2-monitor.service
     103ms phpsessionclean.service
      91ms srv-dev\x2ddisk\x2dby\x2duuid\x2d1d5ab1fe\x2d926c\x2d449b\x2daaef\x2d3668de0848c7.mount
      90ms proc-fs-nfsd.mount
      86ms smbd.service
      84ms php8.2-fpm.service
      82ms srv-dev\x2ddisk\x2dby\x2duuid\x2d152040a4\x2de1f5\x2d4944\x2dbde5\x2d0652ea0edaa1.mount
      78ms srv-dev\x2ddisk\x2dby\x2duuid\x2d0108ae5d\x2d861b\x2d4524\x2d80d7\x2d0d708e6e2d73.mount
      69ms srv-dev\x2ddisk\x2dby\x2duuid\x2d74485af4\x2d9495\x2d4e4e\x2d8250\x2db560f7e07de1.mount
      69ms openmediavault-engined.service
      65ms systemd-resolved.service
      63ms srv-dev\x2ddisk\x2dby\x2duuid\x2d32e967a1\x2d07a2\x2d41b0\x2da546\x2df0aaa2764952.mount
      63ms systemd-udev-trigger.service
      59ms samba-ad-dc.service
      53ms chrony.service
      47ms nut-driver-enumerator.service
      47ms systemd-logind.service
      46ms modprobe@drm.service
      41ms keyboard-setup.service
      40ms systemd-modules-load.service
      39ms monit.service
      33ms rrdcached.service
      32ms systemd-udevd.service
      29ms networking.service
      29ms ssh.service
      24ms avahi-daemon.service
      21ms srv-mergerfs-Schweinshausen_Pool.mount
      20ms systemd-tmpfiles-clean.service
      20ms dev-hugepages.mount
      19ms e2scrub_reap.service
      19ms dev-mqueue.mount
      19ms systemd-sysusers.service
      18ms sys-kernel-debug.mount
      18ms sys-kernel-tracing.mount
      17ms docker.socket
      16ms kmod-static-nodes.service
      15ms modprobe@configfs.service
      15ms nginx.service
      15ms modprobe@dm_mod.service
      14ms rsyslog.service
      14ms modprobe@efi_pstore.service
      14ms dev-disk-by\x2duuid-72063b11\x2d8357\x2d409e\x2d9b6c\x2defe78baaca6a.swap
      14ms collectd.service
      14ms modprobe@fuse.service
      13ms systemd-tmpfiles-setup.service
      13ms wpa_supplicant.service
      13ms modprobe@loop.service
      12ms boot-efi.mount
      12ms systemd-tmpfiles-setup-dev.service
      11ms systemd-update-utmp-runlevel.service
      11ms nfs-mountd.service
       9ms dbus.service
       9ms systemd-network-generator.service
       9ms cpufrequtils.service
       9ms systemd-remount-fs.service
       9ms systemd-binfmt.service
       7ms openmediavault-firewall.service
       7ms systemd-random-seed.service
       7ms rpc-statd.service
       6ms systemd-update-utmp.service
       5ms export-Schweinshausen_Pool.mount
       5ms console-setup.service
       5ms systemd-sysctl.service
       4ms systemd-user-sessions.service
       4ms rpcbind.service
       4ms proc-sys-fs-binfmt_misc.mount
       4ms sys-fs-fuse-connections.mount
       4ms nfs-idmapd.service
       3ms run-rpc_pipefs.mount
       3ms systemd-quotacheck.service
       3ms openmediavault-cleanup-monit.service
       3ms sys-kernel-config.mount
       3ms openmediavault-cleanup-php.service
       2ms nfsdcld.service
       2ms rpc-statd-notify.service
       2ms ifupdown-pre.service
       1ms tmp.mount
       1ms postfix.service
      13us blk-availability.service

Display More

This is the output of the command. What can I do with it or what does it tell me?

Soma · Dec 19th 2024

Quote from bennySB

what does it tell me?

It shows that your network takes a LOOONNNGG time to come up and that might be the reason for the errors on the rest of the services
The reason, you will have to find for yourself.

Maybe a search on the web will give you better answers.

For me, I would just disable those notifications on the GUI and be done with it (I did it since the Pi era):

Nonetheless, if you still want to monitor those settings, you'll have to wait for some better idea or search for yourself.
The first thing is to check router settings, NIC settings, cables to name a few

bennySB · Dec 19th 2024

Thank you for your feedback.

It can't really be the network. I can log in again via SSH a few seconds after the power on, but the WebUI takes a lot longer to be available.

Only after the WebUI and therefore OMV itself is available do the error messages appear by email.

So the network is available, but the services still won't start, but why?

I deactivated the process monitoring in the notification settings and now “only” the error message regarding nginx appears after a restart

macom · Dec 20th 2024

Do you have more than one network adapter?

bennySB · Dec 21st 2024

enp2s0 is my real Ethernet connector and enp2s0.10 a VLAN connector for my docker mcvlan.

macom · Dec 21st 2024

Quote from bennySB

enp2s0.10 a VLAN connector for my docker mcvlan

Did you add it manually?

bennySB · Dec 21st 2024

What do you mean with manually?

I added it over the WebUI of OMV.

macom · Dec 22nd 2024

It is probably the one that takes long to come up. Maybe you can disable IPv6 on that adapter.

bennySB · Dec 22nd 2024

Thank you for the tip, that was exactly the right point.

I also think I know why there is a problem with IPv6.

Quite simply: I have deactivated IPv6 in my DHCP server. OMV is assigned a v6 address, but probably only a standard address and therefore cannot go online.

Thank you once again.

Fresh OMV 7 installation --> rrdcached and nginx alerts

macom Dec 12th 2024

bennySB Dec 22nd 2024

Participate now!