Nginx reverse proxy broken after latest OMV 5 update (v5.6.21)

  • Hi,

    I'm facing a reverse proxy problem since this morning.
    I have a qbittorrent-nox server running on my NAS, listening on port 8080 for the WebUI.
    I setup a reverse proxy on nginx in order to access it from the internet with an URL on port 443.


    It used to work fine but until the last update (v5.6.21) this morning. I have checked the changelog but nothing seems related.

    Anyway, when I try to access https://torrent.myInternetURL.fr, I have the OMV WebUI page instead of the qBittorrent page, like no redirection is done.
    If I try to access directly the qBittorrent WebUI with http://torrent.myInternetURL.fr:8080, I have the correct page.

    I can't figure out what could have changed to brake te redirection...
    Any Web server expert here ?

    Thanks

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • To be complete, this configuration still works fine on another OMV server not updated yet !

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • Instead of editing files on the host and to prevent this same situation to happen again:


    Fire a SWAG docker creating a certificate for your WAN domain.


    use the proxy-conf for qBittorrent (either subfolder or subdomain) and change this:

    Code
    set $upstream_app qbittorrent;

    to this:

    Code
    set $upstream_app 192.168.x.x; #<--- Lan IP of where qBittorrent is running.


    Problem solved


    You can even use a "homebrew" proxy-conf for OMV5.

  • Hummm... I don't want to use docker for several reasons, and it work fine since more than 1 year !

    Any other suggestions ?

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • To be complete, this configuration still works fine on another OMV server not updated yet !

    Any other suggestions ?

    Other than suggest you to find the diffs of the files that are changed before and after the update:

    Sorry, don't have any other.

  • OMV Update :


    Latest known got access :


    tail -f qbittorrent-webgui_access.log

    Code
    ::ffff:91.212.21.220 - - [02/Dec/2021:09:03:54 +0100] "POST /api/v2/rss/items?kwonl09y HTTP/2.0" 200 65605 "https://torrent.myInternetURL.fr/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.20 Safari/537.36 Edg/97.0.1072.13"

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • Anyway to rollback and re-install openmediavault v5.6.20 ?

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • Pretty sure it's an easy task but don't know how.


    Maybe the same way you install a previous version of a deb:

    How To Downgrade Packages To A Specific Version With Apt In Debian, Ubuntu Or Linux Mint - Linux Uprising Blog


    Please, DO NOT follow my info without other people's confirmation.

    It might do more harm than benefit.


    Might be better if you can view the working OMV nginx files and compare to the non-working one.

  • As your config did not change, can you please check if your file is really included in the nginx config file.

    nginx -c /pat/to/mginx/config -T | more will check the config and dump the result. Hit <Enter> to scroll through the pages.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • As your config did not change, can you please check if your file is really included in the nginx config file.

    nginx -c /pat/to/mginx/config -T | more will check the config and dump the result. Hit <Enter> to scroll through the pages.

    Thanks for the tip... unfortunately, after verification, the conf file is included.

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • Can you compare the config dump with what you have in the non-updated systems?


    What you can do to debug is to run nginx in no daemon mode with excessive logging to find out why these requests are routed to OMV instead of the reverse proxy.


    Stop nginx service from the command line

    start nginx with

    Code
    /usr/sbin/nginx -g "daemon off;error_log /dev/stdout debug;"

    look at the logs . if nothing is found, you have to enable debugg logging in the server parts of the config.


    Please check the cli flags with the docs, it is written from memory.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • Just updated the other server and reverse proxy is broken too with v5.6.21... so for sure, this is related to the update.

    So, started nginx in cli mode, and everything is normal... no error code. I'm lost !


    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

  • You do not find anything in the error log, because nginx does not consider it an error, but redirects to the main server.

    Did you have a chance to compare the nginx -T between working and non working servers?

    Can you compare the versions of nginx between working and non working? (nginx -v)

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • You do not find anything in the error log, because nginx does not consider it an error, but redirects to the main server.

    Did you have a chance to compare the nginx -T between working and non working servers?

    Can you compare the versions of nginx between working and non working? (nginx -v)

    Nginx -T are the same... so same broken reverse proxy.
    Nginx is v1.14.2, the latest package from debian repo.

    The fact that I'm sure is :
    - they have exactly the same nginx/proxy config files.
    - this morning, I update from v5.6.20 to v5.6.21 the first server; reverse proxy stopped just after (<1 min).
    - this afternoon, I updated the second server, reverse proxy stopped just after (<1 min).

    Log file from QB access and error are empty since the update, and the access are logged in OMV access log file....


    OMV access log file :

    Code
    Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.20 Safari/537.36 Edg/97.0.1072.13"
    91.212.21.220 - - [02/Dec/2021:15:18:49 +0100] "GET / HTTP/1.1" 200 1279 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.20 Safari/537.36 Edg/97.0.1072.13"
    127.0.0.1 - - [02/Dec/2021:15:18:53 +0100] "GET / HTTP/1.1" 200 3436 "-" "Monit/5.26.0"
    127.0.0.1 - - [02/Dec/2021:15:18:53 +0100] "GET / HTTP/1.1" 200 3436 "-" "Monit/5.26.0"


    OMV error log file

    Code
    91.212.21.220 - - [02/Dec/2021:15:25:25 +0100] "GET /favicon.ico HTTP/1.1" 200 1406 "https://torrent.myURL.fr/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.20 Safari/537.36 Edg/97.0.1072.13"
    127.0.0.1 - - [02/Dec/2021:15:25:54 +0100] "GET / HTTP/1.1" 200 3436 "-" "Monit/5.26.0"
    127.0.0.1 - - [02/Dec/2021:15:25:54 +0100] "GET / HTTP/1.1" 200 3436 "-" "Monit/5.26.0"

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

    Einmal editiert, zuletzt von sbocquet ()

    • Offizieller Beitrag

    The fact that I'm sure is :
    - they have exactly the same nginx/proxy config files.
    - this morning, I update from v5.6.20 to v5.6.21 the first server; reverse proxy stopped just after (<1 min).
    - this afternoon, I updated the second server, reverse proxy stopped just after (<1 min).

    If you have not pressed the Apply button in the UI then nothing should have happened. The latest version 5.6.21 has marked the nginx, host and postfix modules as dirty, thus their configuration is re-deployed on pressing Apply. But there are no changes in the code, except that the IPv6 detection has been improved/fixed, thus it can happen that now only IPv4 configuration is generated if IPv6 is not configured.

  • Thanks votdev for the hint. Lookin in the generated config nginx only listenes to IPv6 address on the proxy:


    Code
    listen [::]:80;
    listen [::]:443 ssl http2;


    Shouldn't there be

    Code
    listen 80;
    listen 443 ssl https; 

    also?

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • If you have not pressed the Apply button in the UI then nothing should have happened. The latest version 5.6.21 has marked the nginx, host and postfix modules as dirty, thus their configuration is re-deployed on pressing Apply. But there are no changes in the code, except that the IPv6 detection has been improved/fixed, thus it can happen that now only IPv4 configuration is generated if IPv6 is not configured.

    Thanks Volker ( votdev), you point me in the right direction once again ;) As you said, no IPv6 was configured in those servers... but the nginx reverse proxy conf file was "misconfigured" !


    Modified the configuration from

    Code
    listen [::]:80;
    listen [::]:443 ssl http2;

    to

    Code
    listen 80;
    listen 443 ssl http2;


    Big thanks both of you for your time.

    Lian Li PC-V354 (with Be Quiet! Silent Wings 3 fans)
    ASRock Rack x470D4U | AMD Ryzen 5 3600 | Crucial 16GB DDR4 2666MHz ECC | Intel x550T2 10Gb NIC

    1 x ADATA 8200 Pro 256MB NVMe for System/Caches/Logs/Downloads
    5 x Western Digital 10To HDD in RAID 6 for Datas
    1 x Western Digital 2To HDD for Backups

    Powered by OMV v5.6.26 & Linux kernel 5.10.x

    Einmal editiert, zuletzt von sbocquet ()

  • sbocquet

    Hat das Label OMV 5.x hinzugefügt.
  • sbocquet

    Hat das Label gelöst hinzugefügt.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!