Can't login to WebGUI

ptruman · 21. Oktober 2017

Well this is awkward

OMV Erasmus 3.0.89 has been running fine - and I have LetsEncrypt running, but I use port 444 for HTTPS, as I have SSH running on 443, so I can proxy in from outside (yes, I'm firewalled, and SSH has three (3) factor auth running)

The other day I installed Asterisk 15, with FreePBX 13. FreePBX uses Apache, but I set that to run on port 81 (and the admin interface was working). I think it did some bits with MySQL, but to be fair, I didn't try to login to OMV as all services are running fine (apparently!)

Tried to login today, and I get the blue WebGUI login screen.
If I put in a valid admin user/pass, it clears, looks like it'll login, then re-appears at the login screen.
If I put in an invalid user/pass, it gives me an error.

I did have lots of collectd errors in syslog, which I've cleared and restarted collectd and that seems happier, but I still can't login....

omv-engined is "already running", omv-firstaid shows no issues, although if I run RRD check, randomly I get told

All RRD database files are valid.
Action failed -- Other action already in progress -- please try again later

If I run it again, I don't get "Action failed", then again, I will.

Not sure if that's the problem - or a red herring?

Any clues?

ptruman · 21. Oktober 2017

Update : RRD is getting errors in syslog still, as seen below....although it only seems to get those if I run the omv-firstaid rrd checker....

You can see my check of omv-engined, then an rrd restart, but then they resume barfing.
What have I/it done?!

The drives are not out of space (df -h below)

Code

Filesystem      Size  Used Avail Use% Mounted on
udev             10M     0   10M   0% /dev
tmpfs           1.2G   16M  1.2G   2% /run
/dev/md0         70G  8.2G   59G  13% /
tmpfs           2.8G  4.0K  2.8G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.8G     0  2.8G   0% /sys/fs/cgroup
tmpfs           2.8G  568K  2.8G   1% /tmp
/dev/md127      459G  285G  151G  66% /sftp/pete/Pete

syslog tail below:

Code

Oct 21 11:44:02 MediaVault crontab[6511]: (asterisk) LIST (asterisk)
Oct 21 11:44:02 MediaVault crontab[6513]: (asterisk) LIST (asterisk)
Oct 21 11:44:02 MediaVault crontab[6515]: (asterisk) REPLACE (asterisk)
Oct 21 11:44:02 MediaVault crontab[6517]: (asterisk) LIST (asterisk)
Oct 21 11:44:18 MediaVault omv-engined[6558]: Daemon already running (pid=1302)
Oct 21 11:45:01 MediaVault cron[962]: (asterisk) RELOAD (crontabs/asterisk)
Oct 21 11:45:01 MediaVault CRON[6602]: (root) CMD (/var/lib/openmediavault/cron.d/userdefined-cc54ed52-5c45-4102-94d5-17cd89c60ca8 >/de$
Oct 21 11:45:01 MediaVault CRON[6603]: (asterisk) CMD ([ -x /var/lib/asterisk/bin/schedtc.php ] && /var/lib/asterisk/bin/schedtc.php)
Oct 21 11:45:01 MediaVault CRON[6605]: (asterisk) CMD (/usr/sbin/fwconsole userman --syncall -q)
Oct 21 11:45:01 MediaVault CRON[6604]: (asterisk) CMD ([ -x /var/www/html/admin/modules/dashboard/scheduler.php ] && /var/www/html/admi$
Oct 21 11:45:01 MediaVault CRON[6606]: (root) CMD (/usr/sbin/omv-mkgraph >/dev/null 2>&1)
Oct 21 11:45:02 MediaVault rrdcached[1130]: Received FLUSHALL
Oct 21 11:45:02 MediaVault crontab[6644]: (asterisk) LIST (asterisk)
Oct 21 11:45:02 MediaVault crontab[6646]: (asterisk) LIST (asterisk)
Oct 21 11:45:02 MediaVault crontab[6649]: (asterisk) REPLACE (asterisk)
Oct 21 11:45:02 MediaVault crontab[6654]: (asterisk) LIST (asterisk)
Oct 21 11:45:02 MediaVault crontab[6658]: (asterisk) LIST (asterisk)
Oct 21 11:45:02 MediaVault crontab[6660]: (asterisk) REPLACE (asterisk)
Oct 21 11:45:02 MediaVault crontab[6664]: (asterisk) LIST (asterisk)
Oct 21 11:45:08 MediaVault monit[1129]: 'rrdcached' stop on user request
Oct 21 11:45:08 MediaVault monit[1129]: Monit daemon with PID 1129 awakened
Oct 21 11:45:08 MediaVault monit[1129]: Awakened by User defined signal 1
Oct 21 11:45:08 MediaVault monit[1129]: 'rrdcached' stop: /bin/systemctl
Oct 21 11:45:08 MediaVault systemd[1]: Stopping LSB: start the RRDtool data caching daemon...
Oct 21 11:45:08 MediaVault rrdcached[1130]: caught SIGTERM
Oct 21 11:45:08 MediaVault rrdcached[1130]: starting shutdown
Oct 21 11:45:08 MediaVault rrdcached[1130]: clean shutdown; all RRDs flushed
Oct 21 11:45:08 MediaVault rrdcached[1130]: removing journals
Oct 21 11:45:08 MediaVault rrdcached[1130]: removing old journal /var/lib/rrdcached/journal/rrd.journal.1508581696.408967
Oct 21 11:45:08 MediaVault rrdcached[1130]: goodbye
Oct 21 11:45:09 MediaVault monit[6817]: Action failed -- Other action already in progress -- please try again later
Oct 21 11:45:09 MediaVault rrdcached[6764]: Stopping RRDtool data caching daemon: rrdcached.
Oct 21 11:45:09 MediaVault systemd[1]: Stopped LSB: start the RRDtool data caching daemon.
Oct 21 11:45:09 MediaVault monit[1129]: 'rrdcached' stop action done
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_update (/var/lib/rrdcached/db/localhost/cpu-0/cpu-user.rrd, [15085827$
Oct 21 11:45:10 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:10 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:10 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:10 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:10 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:10 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:10 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:15 MediaVault monit[1129]: 'rrdcached' stop on user request
Oct 21 11:45:15 MediaVault monit[1129]: Monit daemon with PID 1129 awakened
Oct 21 11:45:15 MediaVault monit[1129]: Awakened by User defined signal 1
Oct 21 11:45:15 MediaVault monit[1129]: 'rrdcached' stop action done
Oct 21 11:45:16 MediaVault systemd[1]: Time has been changed
Oct 21 11:45:16 MediaVault monit[1129]: 'rrdcached' start on user request
Oct 21 11:45:16 MediaVault monit[1129]: Monit daemon with PID 1129 awakened
Oct 21 11:45:16 MediaVault monit[1129]: Awakened by User defined signal 1
Oct 21 11:45:16 MediaVault monit[1129]: 'rrdcached' start: /bin/systemctl
Oct 21 11:45:16 MediaVault systemd[1]: Starting LSB: start the RRDtool data caching daemon...
Oct 21 11:45:16 MediaVault rrdcached[6883]: Starting RRDtool data caching daemon: rrdcached.
Oct 21 11:45:16 MediaVault systemd[1]: Started LSB: start the RRDtool data caching daemon.
Oct 21 11:45:16 MediaVault rrdcached[6887]: starting up
Oct 21 11:45:16 MediaVault rrdcached[6887]: checking for journal files
Oct 21 11:45:16 MediaVault rrdcached[6887]: started new journal /var/lib/rrdcached/journal/rrd.journal.1508582716.729804
Oct 21 11:45:16 MediaVault rrdcached[6887]: journal processing complete
Oct 21 11:45:16 MediaVault rrdcached[6887]: listening for connections
Oct 21 11:45:16 MediaVault monit[1129]: 'rrdcached' start action done
Oct 21 11:45:29 MediaVault monit[1129]: 'rrdcached' stop on user request
Oct 21 11:45:29 MediaVault monit[1129]: Monit daemon with PID 1129 awakened
Oct 21 11:45:29 MediaVault monit[1129]: Awakened by User defined signal 1
Oct 21 11:45:29 MediaVault monit[1129]: 'rrdcached' stop: /bin/systemctl
Oct 21 11:45:29 MediaVault systemd[1]: Stopping LSB: start the RRDtool data caching daemon...
Oct 21 11:45:29 MediaVault rrdcached[6887]: starting shutdown
Oct 21 11:45:29 MediaVault rrdcached[6887]: clean shutdown; all RRDs flushed
Oct 21 11:45:29 MediaVault rrdcached[6887]: removing journals
Oct 21 11:45:29 MediaVault rrdcached[6887]: removing old journal /var/lib/rrdcached/journal/rrd.journal.1508582716.729804
Oct 21 11:45:29 MediaVault rrdcached[6887]: goodbye
Oct 21 11:45:30 MediaVault monit[6960]: Action failed -- Other action already in progress -- please try again later
Oct 21 11:45:30 MediaVault rrdcached[6908]: Stopping RRDtool data caching daemon: rrdcached.
Oct 21 11:45:30 MediaVault systemd[1]: Stopped LSB: start the RRDtool data caching daemon.
Oct 21 11:45:30 MediaVault monit[1129]: 'rrdcached' stop action done
Oct 21 11:45:30 MediaVault collectd[1259]: rrdcached plugin: rrdc_update (/var/lib/rrdcached/db/localhost/cpu-0/cpu-user.rrd, [15085827$
Oct 21 11:45:30 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:30 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:30 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:30 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:30 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:30 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:30 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:30 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.
Oct 21 11:45:30 MediaVault collectd[1259]: Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed wit$
Oct 21 11:45:30 MediaVault collectd[1259]: rrdcached plugin: rrdc_connect (unix:/var/run/rrdcached.sock) failed with status 2.

Alles anzeigen

In case something is blocking a port, here is netstatn -tulpen : https://pastebin.com/aZHsuJrL
(note that I run Java for CrashPlan and OpenRemote)

ptruman · 21. Oktober 2017

Right - god knows why I didn't check the nginx logs, but this is hiding in there:

Code

2017/10/21 19:01:23 [error] 12549#0: *15 FastCGI sent in stderr: "PHP message: PHP Warning:  session_start(): open(/var/lib/php5/sessions/sess_cg9bvqt33fvfr22h449cn4f703, O_RDWR) failed: Permission denied (13) in /usr/share/php/openmediavault/session.inc on line 43" while reading response header from upstream, client: 127.0.0.1, server: openmediavault-webgui, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm-openmediavault-webgui.sock:", host: "127.0.0.1:444"

Clues?

ptruman · 21. Oktober 2017

Got in, by changing /var/lib/php5/* to root (chown & chgrp)
Also ran:

chmod 1733 /var/lib/php5/sessions
chmod 0755 /var/lib/php5/modules

Jetzt mitmachen!