Upgrade to 3 - Everything was working, but now receiving 502 Bad Gateway!

RS1 · 28. Dezember 2016

Hi there, I upgraded OMV2 to 3 using omv-release-upgrade. I had a couple of issues with some unsupported plugins, however, after searching the forums, figured out how to uninstall them and I could login just fine. Since then, I installed all of the updates and have been cleaning up my data.

I just rebooted and I receive the 502 Gateway error. However, I can login via SSH and into Webmin just fine. The output from dpkg -l | grep openm is:

Code

ii openmediavault 3.0.58 all Open network attached storage solution
ii openmediavault-autoshutdown 3.1.2 all OpenMediaVault AutoShutdown Plugin
ii openmediavault-clamav 3.2.7 all OpenMediaVault ClamAV plugin
ii openmediavault-keyring 1.0 all GnuPG archive keys of the OpenMediaVault archive
ii openmediavault-minidlna 3.3.4 all OpenMediaVault miniDLNA (DLNA server) plugin
ii openmediavault-netatalk 3.2.5 all OpenMediaVault netatalk (AppleTalk Protocol Suite) plugin
ii openmediavault-nzbget 3.2 all command-line based binary newsgrabber for nzb files
ii openmediavault-omvextrasorg 3.4.12 all OMV-Extras.org Package Repositories for OpenMediaVault
ii openmediavault-openvpn 3.0.4 all OpenVPN plugin for OpenMediaVault.
ii openmediavault-sensors 3.0 all Provides a cpu temperature monitor
ii openmediavault-sickbeard 3.2.1 all OpenMediaVault Sick Beard plugin
ii openmediavault-syncthing 3.8 all Syncthing plugin for OpenMediaVault.
ii openmediavault-unionfilesystems 3.1.9 all Union filesystems plugin for OpenMediaVault.
ii openmediavault-usbbackup 3.2.12 all OpenMediaVault USB/eSATA backup plugin
ii openmediavault-wol 3.1 all OpenMediaVault WOL plugin

Alles anzeigen

Everything has come to a standstill so if anyone can help me through this issue, it would be greatly appreciated.

Wolf2000 · 28. Dezember 2016

openmediavault-keyring 1.0 is problem

ryecoaaron · 28. Dezember 2016

keyring 1.0 is the latest version and isn't a plugin.

try:
omv-mkconf nginx
omv-mkconf php5fpm
systemctl restart nginx
systemctl restart php5-fpm

RS1 · 28. Dezember 2016

Zitat von ryecoaaron

keyring 1.0 is the latest version and isn't a plugin.

try:
omv-mkconf nginx
omv-mkconf php5pm
systemctl restart nginx
systemctl restart php5-fpm

Alles anzeigen

Thanks for the response. Upon executing the second command, I receive the following error:

Code

ERROR: The script '/usr/share/openmediavault/mkconf/php5pm' does not exist

Upon executing the last command, I receive the following:

Code

Job for php5-fpm.service failed. See 'systemctl status php5-fpm.service' and 'journalctl -xn' for details.

I presume the service cannot restart as it doesn't exist right? Thoughts?

RS1 · 28. Dezember 2016

I'm really trying to figure this out but unfortunately my knowledge is very limited. I executed: ps aux | grep php

Output: root 1791 0.0 0.0 12728 2156 pts/0 S+ 19:59 0:00 grep php

So I guess the process is running. But for some reason, I can't restart the process. I also don't know how to 'see' systemct1 status php-fpm.service; or journalctl -xn

I'm running out of ideas here..

ryecoaaron · 28. Dezember 2016

i had a typo in fpm command since i am typing on my phone. I edited my post. try the commands again

RS1 · 28. Dezember 2016

The first 3 commands executed fine; however, I still get the same error message after executing the last command:

Code

Job for php5-fpm.service failed. See 'systemctl status php5-fpm.service' and 'journalctl -xn' for details.

ryecoaaron · 29. Dezember 2016

What is the output of the commands it tells you to look at for details?

RS1 · 29. Dezember 2016

My linux knowledge is quite basic - how do I retrieve those details?

Thanks

RS1 · 29. Dezember 2016

Zitat von ryecoaaron

What is the output of the commands it tells you to look at for details?

Okay.. I've figured out how to reveal the status of those files:

For journalctl -xn:

Code

root@NAS:/# journalctl -xn
-- Logs begin at Wed 2016-12-28 20:15:20 GMT, end at Wed 2016-12-28 23:44:43 GMT. --
Dec 28 23:44:13 NAS systemd[1]: Unit php5-fpm.service entered failed state.
Dec 28 23:44:43 NAS monit[1125]: 'php5-fpm' failed to start (exit status 1) -- /bin/systemctl: Job for php5-fpm.service failed. See '
systemctl status php5-fpm.service' and 'journalctl -xn' for details.
Dec 28 23:44:43 NAS monit[1125]: 'nginx' failed protocol test [HTTP] at INET[127.0.0.1:80] via TCP -- HTTP error: Server returned sta
tus 502
Dec 28 23:44:43 NAS monit[1125]: 'nginx' trying to restart
Dec 28 23:44:43 NAS monit[1125]: 'nginx' stop: /bin/systemctl
Dec 28 23:44:43 NAS systemd[1]: Stopping A high performance web server and a reverse proxy server...
-- Subject: Unit nginx.service has begun shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nginx.service has begun shutting down.
Dec 28 23:44:43 NAS systemd[1]: Stopped A high performance web server and a reverse proxy server.
-- Subject: Unit nginx.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nginx.service has finished shutting down.
Dec 28 23:44:43 NAS monit[1125]: 'nginx' start: /bin/systemctl
Dec 28 23:44:43 NAS systemd[1]: Starting A high performance web server and a reverse proxy server...
-- Subject: Unit nginx.service has begun with start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nginx.service has begun starting up.
Dec 28 23:44:43 NAS systemd[1]: Started A high performance web server and a reverse proxy server.
-- Subject: Unit nginx.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nginx.service has finished starting up.
-- 
-- The start-up result is done.

Alles anzeigen

For php5-fpm:

Code

● php5-fpm.service - The PHP FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php5-fpm.service; enabled)
   Active: failed (Result: exit-code) since Wed 2016-12-28 23:48:15 GMT; 56s ago
  Process: 12649 ExecStart=/usr/sbin/php5-fpm --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.conf (code=exited, status=78)
  Process: 12644 ExecStartPre=/usr/lib/php5/php5-fpm-checkconf (code=exited, status=0/SUCCESS)
 Main PID: 12649 (code=exited, status=78)


Dec 28 23:48:15 NAS php5-fpm[12649]: [28-Dec-2016 23:48:15] ERROR: [pool openmediavault-extplorer] cannot get uid for user 'extplorer'
Dec 28 23:48:15 NAS php5-fpm[12649]: [28-Dec-2016 23:48:15] ERROR: FPM initialization failed
Dec 28 23:48:15 NAS systemd[1]: php5-fpm.service: main process exited, code=exited, status=78/n/a
Dec 28 23:48:15 NAS systemd[1]: Failed to start The PHP FastCGI Process Manager.
Dec 28 23:48:15 NAS systemd[1]: Unit php5-fpm.service entered failed state.

Alles anzeigen

Interestingly, in the second output - there is a reference to the user 'extplorer'. I deleted this user from WebUI since I uninstalled the plugin in OMV3. I'm not sure whether this is related to the issue that I am having.

Thanks

ryecoaaron · 29. Dezember 2016

Looks like the extplorer pool is still there.

rm -f /etc/php5/fpm/pool.d/openmediavault-extplorer-site.conf /etc/nginx/openmediavault-webgui.d/openmediavault-extplorer-site.conf

Then execute the commands I posted above again.

RS1 · 29. Dezember 2016

Success! Thanks again, much appreciated!

RS1 · 29. Dezember 2016

Sidenote: for anyone else who has a difficult time navigating linux, I found the following useful: https://www.digitalocean.com/c…ystemd-services-and-units

c0m4r · 6. Januar 2017

Hi
I've read whole thread as i have very similar problem: i have perform omv upgrade to 3.x and have 502 bad gateway. I did :

Code

omv-mkconf nginx
omv-mkconf php5pm
systemctl restart nginx
systemctl restart php5-fpm

but last command return errors:

Code

root@Santiago:~# systemctl restart php5-fpm
Job for php5-fpm.service failed. See 'systemctl status php5-fpm.service' and 'journalctl -xn' for details.

and then:

Code

root@Santiago:~#  journalctl -xn
sty 06 20:48:39 Santiago systemd[1]: Starting The PHP FastCGI Process Manager...
-- Subject: Unit php5-fpm.service has begun with start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit php5-fpm.service has begun starting up.
sty 06 20:48:40 Santiago php5-fpm[21598]: [06-Jan-2017 20:48:40] ERROR: [pool 6f6fec40-5f56-4564-a9ea-ae7617405449] please specify user and group other than root
sty 06 20:48:40 Santiago php5-fpm[21598]: [06-Jan-2017 20:48:40] ERROR: FPM initialization failed
sty 06 20:48:40 Santiago systemd[1]: php5-fpm.service: main process exited, code=exited, status=78/n/a
sty 06 20:48:40 Santiago systemd[1]: Failed to start The PHP FastCGI Process Manager.
-- Subject: Unit php5-fpm.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit php5-fpm.service has failed.
--
-- The result is failed.
sty 06 20:48:40 Santiago systemd[1]: Unit php5-fpm.service entered failed state.

Alles anzeigen

Code

root@Santiago:~# systemctl status php5-fpm.service
● php5-fpm.service - The PHP FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php5-fpm.service; enabled)
   Active: failed (Result: exit-code) since pią 2017-01-06 20:41:54 CET; 37s ago
  Process: 21038 ExecStart=/usr/sbin/php5-fpm --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.conf (code=exited, status=78)
  Process: 21034 ExecStartPre=/usr/lib/php5/php5-fpm-checkconf (code=exited, status=0/SUCCESS)
 Main PID: 21038 (code=exited, status=78)
sty 06 20:41:54 Santiago php5-fpm[21038]: [06-Jan-2017 20:41:54] ERROR: [pool 6f6fec40-5f56-4564-a9ea-ae7617405449] please specify user and group other than root
sty 06 20:41:54 Santiago php5-fpm[21038]: [06-Jan-2017 20:41:54] ERROR: FPM initialization failed
sty 06 20:41:54 Santiago systemd[1]: php5-fpm.service: main process exited, code=exited, status=78/n/a
sty 06 20:41:54 Santiago systemd[1]: Failed to start The PHP FastCGI Process Manager.
sty 06 20:41:54 Santiago systemd[1]: Unit php5-fpm.service entered failed state.

Alles anzeigen

I assume that problem is with "ERROR: [pool 6f6fec40-5f56-4564-a9ea-ae7617405449] please specify user and group other than root" but i have no idea where and which user should i change ? Could you please help ?

RS1 · 6. Januar 2017

I personally had several issues when I upgraded from 2.x to 3.x. My issues were resolved when I did a fresh install of 3.x. I'm sure that the issues could be fixed one at a time, but in retrospect - it was much, much quicker (and less stressful) to do a fresh install.

c0m4r · 6. Januar 2017

I know but i have also many others services on this machine and i dont like to reinstall whole stuff Additionally i feel that i am very close to finish this upgrade successfully if i could manage to overcome mentioned above problem.

EDIT:
okay, i edited /lib/systemd/system/php5-fpm.service :
ExecStart=/usr/sbin/php5-fpm --allow-to-run-as-root --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.confand added "--allow-to-run-as-root" which enabled me to restart php5-fm.

ryecoaaron · 6. Januar 2017

Zitat von c0m4r

I know but i have also many others services on this machine and i dont like to reinstall whole stuff

Good thing we don't use Redhat/CentOS.

Zitat von c0m4r

EDIT:
okay, i edited /lib/systemd/system/php5-fpm.service :
ExecStart=/usr/sbin/php5-fpm --allow-to-run-as-root --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.confand added "--allow-to-run-as-root" which enabled me to restart php5-fm.

This is a bad idea and avoids fixing the problem. What is the output of: dpkg -l | grep openm

c0m4r · 7. Januar 2017

Hi
I managed to get webgui working (i had to uninstall some plugins, even omvextras) but apparently i have some popup with error :

Code

Błąd #0:
exception 'OMV\AssertException' with message 'The property 'user.0' does not exist in the model 'conf.service.phpfpm.pool'.' in /usr/share/php/openmediavault/config/configobject.inc:71
Stack trace:
#0 /usr/share/php/openmediavault/config/configobject.inc(186): OMV\Config\ConfigObject->assertExists('user.0')
#1 /usr/share/php/openmediavault/config/configobject.inc(271): OMV\Config\ConfigObject->set('user.0', 'www-data', false)
#2 /usr/share/php/openmediavault/config/configobject.inc(233): OMV\Config\ConfigObject->setFlatAssoc(Array, false, false)
#3 /usr/share/php/openmediavault/config/database.inc(85): OMV\Config\ConfigObject->setAssoc(Array, false)
#4 /usr/share/openmediavault/engined/rpc/phpfpm-server.inc(75): OMV\Config\Database->get('conf.service.ph...')
#5 [internal function]: OMV\Engined\Rpc\PhpFpmServer->getList(Array, Array)
#6 /usr/share/php/openmediavault/rpc/serviceabstract.inc(124): call_user_func_array(Array, Array)
#7 /usr/share/php/openmediavault/rpc/rpc.inc(84): OMV\Rpc\ServiceAbstract->callMethod('getList', Array, Array)
#8 /usr/sbin/omv-engined(516): OMV\Rpc\Rpc::call('PhpFpm', 'getList', Array, Array, 1)
#9 {main}
OK
Ukryj szczegóły

Alles anzeigen

Outputof dpkg -l | grep openm :

Code

root@Santiago:~# dpkg -l | grep openm
ii  openmediavault                                                   3.0.59                               all          Open network attached storage solution
ii  openmediavault-backup                                            3.3.5                                all          backup plugin for OpenMediaVault.
rc  openmediavault-extplorer                                         1.3.1                                all          OpenMediaVault eXtplorer plugin
ii  openmediavault-flashmemory                                       3.4.3                                all          folder2ram plugin for OpenMediaVault
ii  openmediavault-keyring                                           1.0                                  all          GnuPG archive keys of the OpenMediaVault archive
rc  openmediavault-links                                             1.5                                  all          OpenMediaVault links plugin
ii  openmediavault-mysql                                             3.0.4                                all          MySQL plugin for OpenMediaVault.
ii  openmediavault-nginx                                             3.0.9                                all          Nginx plugin for OpenMediaVault.
ii  openmediavault-nut                                               3.2.7                                all          OpenMediaVault Network UPS Tools (NUT) plugin
ii  openmediavault-omvextrasorg                                      3.4.14                               all          OMV-Extras.org Package Repositories for OpenMediaVault
ii  openmediavault-plexmediaserver                                   3.7                                  all          OpenMediaVault Plex Media Server plugin

Alles anzeigen

What do you suggest to should be done with php5-fm error insted of starting it with root user ?

ryecoaaron · 7. Januar 2017

Your error is probably caused the the nginx plugin not updating config.xml properly. I would change the field user to username in the phpfpm section of the nginx plugin.

c0m4r · 7. Januar 2017

Where can i find this section (phpfm) in nginx plugin ? I have installed nextcloud before upgrade, it was and it is working but i cannot edit nor add new pool in nginx plugin via webgui because of this error. Could you please give me a hint where can i repair it ?

Jetzt mitmachen!