kubernetes-dashboard-kong not starting

Keif888 · Jun 19th 2025

Hi,

I have had a problem with the kubernetes-dashboard-kong pod going into a CrashLoopBackOff status with the log output as follows:

Code

root@nas: ~# kubectl logs  kubernetes-dashboard-kong-79867c9c48-zg674 -n kubernetes-dashboard
Defaulted container "proxy" out of: proxy, clear-stale-pid (init)
2025/06/19 08:09:07 [warn] 1#0: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /kong_prefix/nginx.conf:7
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /kong_prefix/nginx.conf:7
2025/06/19 08:09:07 [notice] 1#0: [lua] init.lua:791: init(): [request-debug] token for request debugging: 5593bbbf-f98e-462b-8e18-d4b364150fe6
2025/06/19 08:09:07 [emerg] 1#0: bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
nginx: [emerg] bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
2025/06/19 08:09:07 [notice] 1#0: try again to bind() after 500ms
2025/06/19 08:09:07 [emerg] 1#0: bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
nginx: [emerg] bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
2025/06/19 08:09:07 [notice] 1#0: try again to bind() after 500ms
2025/06/19 08:09:07 [emerg] 1#0: bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
nginx: [emerg] bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
2025/06/19 08:09:07 [notice] 1#0: try again to bind() after 500ms
2025/06/19 08:09:07 [emerg] 1#0: bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
nginx: [emerg] bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
2025/06/19 08:09:07 [notice] 1#0: try again to bind() after 500ms
2025/06/19 08:09:07 [emerg] 1#0: bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
nginx: [emerg] bind() to unix:/kong_prefix/sockets/we failed (98: Address already in use)
2025/06/19 08:09:07 [notice] 1#0: try again to bind() after 500ms
2025/06/19 08:09:07 [emerg] 1#0: still could not bind()
nginx: [emerg] still could not bind()

Display More

This appears to be because the port that is being used for KONG_ADMIN_LISTEN in the default deployment is 8444 which is already in use as per this issue https://github.com/kubernetes/dashboard/issues/8765

I manually worked around this by updating my deployment to use port 8445 and using kubectl apply to push the update through. I couldn't work out how to run the helm command mentioned in the issue above.

Is it possible to update the kubernetes plugin to apply the fix noted in the issue above?

Or update the deployment to change the port away from 8444?

Thanks.

votdev · Jun 19th 2025

Hmmm, it looks like they changed something in the meanwhile.

Maybe this is the trigger to switch from Kubernetes Dashboard to Rancher as the default OMV Kubernetes UI.

Keif888 · Jun 20th 2025

I ended up updating my kubernetes-dashboard-kong deployment and turning the kong admin off as shown below, as changing the port failed on the next reboot.

Code

      dnsPolicy: ClusterFirst
      initContainers:
        - command:
            - rm
            - '-vrf'
            - $KONG_PREFIX/pids
          env:
            - name: KONG_ADMIN_ACCESS_LOG
              value: /dev/stdout
            - name: KONG_ADMIN_ERROR_LOG
              value: /dev/stderr
            - name: KONG_ADMIN_GUI_ACCESS_LOG
              value: /dev/stdout
            - name: KONG_ADMIN_GUI_ERROR_LOG
              value: /dev/stderr
            - name: KONG_ADMIN_LISTEN
              value: 'off'

Display More

votdev · Jun 22nd 2025

Quote from Keif888

the default deployment is 8444 which is already in use as per this issue

Have you changed the Dashboard port from 4443 to 8444 in the Settings tab?

Keif888 · Jun 23rd 2025

No, my settings in the Kubernetes setting tab are:

Enabled - Yes

Datastore - SQLite

Snapshots - a folder that is only used for snapshots

Certificate - Yes, generated with openssl

Load Balancer

http - 8080

https - 8443

Dashboard

port - 4443

I have setup a script that runs on reboot (all within OMV using the Scripts plugin) as follows, as the kubernetes-dashboard-kong deployment never works after reboot. But if I scale it down and back up, it then works. I have no idea what the underlying cause is though.

Bash

#!/bin/bash
sleep 600
if journalctl --since "65 seconds ago" SYSLOG_IDENTIFIER=k3s -g "restarting failed container=proxy pod=kubernetes-dashboard-kong" | grep k3s; then
    kubectl scale -n kubernetes-dashboard deployment kubernetes-dashboard-kong --replicas=0
    sleep 15
    kubectl scale -n kubernetes-dashboard deployment kubernetes-dashboard-kong --replicas=1
fi

votdev · Jun 23rd 2025

Fixed with https://github.com/openmediava…cf40029e0ce475bf5a86734e5 in openmediavault-k8s 7.4.10.

votdev · Jun 23rd 2025

Hmm, it looks like the problem still exists. I have no idea what is going wrong here. If the Dashboard is deployed it is working, but after a reboot of the system it does no work anymore and the container is restarted allover because of the Address already in use.

votdev · Jun 23rd 2025

Commit https://github.com/openmediava…8001f442150c4689826642a90 should finally fix it.

kubernetes-dashboard-kong not starting

crashtest Jun 19th 2025

votdev Jun 23rd 2025

votdev Jun 23rd 2025

votdev Jun 23rd 2025

Participate now!