help to access my first kubernetes container

jata1 · Oct 6th 2024

Hi all,

Out of interest in kubernetes and the plugin, I have installed on my omv 7 rpi4 (test sbc) and kubernetes is running fine.

I created a ssl certificate in OMV GUI and used rpi4.lan as the common name for the certificate. This is configured in the plugin settings.

I can access the kubernetes dashboard on either https://ip_address:4443 or https://rpi4.lan:4443

I used the recipe templates and deployed wetty and filebrowser. Both seem to be running OK.

My issue is that I can't access either of them from https://wetty.rpi4.lan:8443 or https://filebrowser.rpi4.lan:8443

I know i'm doing something silly but not sure what to do to investigate

jata1 · Oct 7th 2024

trying to work out what is going on but I think it might have something to do with these logs that I have found in the traefik pod/container.

I have tried deploying using 'host-selfsigned-cert' and 'host-imported-cert'

Any help appreciated...

Code

time="2024-10-07T04:40:52Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" namespace=filebrowser-app providerName=kubernetescrd ingress=filebrowser-websecure
time="2024-10-07T04:40:52Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" providerName=kubernetescrd namespace=wetty-app ingress=wetty-websecure

votdev · Oct 7th 2024

Please have a look here; it seems there need to be done some additional changes on Raspi systems. Please report back if this fixes the issues.

Post

RE: K8s pluging don't start on Raspberry pi 5

You should have a look at https://docs.k3s.io/installation/requirements?os=pi. I have not added that to the openmediavault-k8s plugin because i was not sure if this till applies. If you can confirm that it works after applying the changes, then i will implement it in the plugin. A reboot of the system is necessary after changing /boot/cmdline.txt.

votdev

Oct 4th 2024

jata1 · Oct 7th 2024

Thanks votdev - very impressive plugin/project. I use docker quite a bit but this in another level.

I already had followed all of this and everything is running fine on my rpi. For some reason, the traefik pod does not like the name of the certificate.

I have looked in resources > certificates and I see it is there - looks like the secretName is correct.

Code

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  annotations:
    objectset.rio.cattle.io/applied: [redacted]
    objectset.rio.cattle.io/id: ""
    objectset.rio.cattle.io/owner-gvk: k3s.cattle.io/v1, Kind=Addon
    objectset.rio.cattle.io/owner-name: openmediavault-cert-manager
    objectset.rio.cattle.io/owner-namespace: kube-system
  creationTimestamp: "2024-10-06T02:09:23Z"
  generation: 1
  labels:
    app.kubernetes.io/part-of: openmediavault
    objectset.rio.cattle.io/hash: 6af49170be28cc50bb270d3baa56c16fcd3a4fb9
  name: host-selfsigned
  namespace: kube-system
  resourceVersion: "1342"
  uid: 86c18281-f7a0-4852-86b0-a78409f58669
spec:
  dnsNames:
  - rpi4.lan
  - '*.rpi4.lan'
  duration: 8760h
  issuerRef:
    kind: ClusterIssuer
    name: selfsigned
  secretName: host-selfsigned-cert
status:
  conditions:
  - lastTransitionTime: "2024-10-06T02:09:25Z"
    message: Certificate is up to date and has not expired
    observedGeneration: 1
    reason: Ready
    status: "True"
    type: Ready
  notAfter: "2025-10-06T02:09:24Z"
  notBefore: "2024-10-06T02:09:24Z"
  renewalTime: "2025-06-06T10:09:24Z"
  revision: 1

Display More

votdev · Oct 7th 2024

Quote from jata1

For some reason, the traefik pod does not like the name of the certificate.

Can you post the log of the Traefik pod?

jata1 · Oct 7th 2024

not much to see in this log...

Code

time="2024-10-07T03:45:09Z" level=info msg="Configuration loaded from flags."
time="2024-10-07T03:45:09Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" ingress=filebrowser-websecure namespace=filebrowser-app providerName=kubernetescrd
time="2024-10-07T03:45:09Z" level=error msg="Error configuring TLS: secret wetty-app/host-imported-cert does not exist" providerName=kubernetescrd ingress=wetty-websecure namespace=wetty-app
time="2024-10-07T03:45:18Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" providerName=kubernetescrd ingress=filebrowser-websecure namespace=filebrowser-app
time="2024-10-07T03:45:18Z" level=error msg="Error configuring TLS: secret wetty-app/host-imported-cert does not exist" namespace=wetty-app providerName=kubernetescrd ingress=wetty-websecure
time="2024-10-07T03:45:18Z" level=error msg="Error configuring TLS: secret wetty-app/host-imported-cert does not exist" providerName=kubernetescrd namespace=wetty-app ingress=wetty-websecure
time="2024-10-07T03:45:18Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" namespace=filebrowser-app providerName=kubernetescrd ingress=filebrowser-websecure
time="2024-10-07T04:39:47Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" namespace=filebrowser-app providerName=kubernetescrd ingress=filebrowser-websecure
time="2024-10-07T04:39:47Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" providerName=kubernetescrd ingress=wetty-websecure namespace=wetty-app
time="2024-10-07T04:40:51Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" ingress=filebrowser-websecure namespace=filebrowser-app providerName=kubernetescrd
time="2024-10-07T04:40:51Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" providerName=kubernetescrd namespace=wetty-app ingress=wetty-websecure
time="2024-10-07T04:40:52Z" level=error msg="Error configuring TLS: secret filebrowser-app/host-imported-cert does not exist" namespace=filebrowser-app providerName=kubernetescrd ingress=filebrowser-websecure
time="2024-10-07T04:40:52Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" providerName=kubernetescrd namespace=wetty-app ingress=wetty-websecure

Display More

jata1 · Oct 8th 2024

So I started again but this time using a clean omv install (VM using KVM plugin on my OMV minipc).

I have the same issue. Installed k8s plugin. Confirmed kubernetes dashboard running. Deployed wetty from the recipe. Seems to deploy fine and I can see the wetty pod running.

I just can't connect using either https://wetty.omv.lan:8443 or http://wetty.omv.lan:8080

Similar situation with the traefik pod log - but not sure if this is the issue (or just me doing something stupid).

traefik pod log:

Code

time="2024-10-08T05:41:56Z" level=info msg="Configuration loaded from flags."
time="2024-10-08T05:45:07Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" namespace=wetty-app providerName=kubernetescrd ingress=wetty-websecure
time="2024-10-08T05:45:07Z" level=error msg="subset not found for wetty-app/wetty" providerName=kubernetescrd ingress=wetty-websecure namespace=wetty-app
time="2024-10-08T05:45:21Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" providerName=kubernetescrd ingress=wetty-websecure namespace=wetty-app
time="2024-10-08T05:48:24Z" level=error msg="Error configuring TLS: secret wetty-app/host-selfsigned-cert does not exist" namespace=wetty-app providerName=kubernetescrd ingress=wetty-websecure

wetty pod log (looks good):

Code

yarn run v1.22.19
$ NODE_ENV=production node . --port=3000 --base=/ --force-ssh --ssh-port=22 --ssh-host=omv
{"base":"/","label":"Wetty","level":"info","message":"Starting server","port":3000,"timestamp":"2024-10-08T05:48:24.979Z","title":"WeTTY - The Web Terminal Emulator"}
{"connection":"http","label":"Wetty","level":"info","message":"Server started","port":3000,"timestamp":"2024-10-08T05:48:24.993Z"}

votdev · Oct 8th 2024

You can ignore the error messages; it is still working.

If you can not reach https://wetty.omv.lan:8443 then your local DNS can not resolve wetty.omv.lan. Make sure the FQDN can be resolved by your DNS. For testing purposes you can add it to /etc/hosts on your host system.

Code

192.xxx.xxx.xxx omv omv.lan filebrowser.omv.lan wetty.omv.lan

Alternatively you can use sslip.io if you do not have the capability to modify your DNS server.

To do so you need to modify the IngressRoute object the following way:

Code

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: immich-websecure
  namespace: immich-app
  ...
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`immich.{{ ipaddr() }}.sslip.io`)
      kind: Rule
      services:
        - name: immich-server
          port: 3001

Display More

After that you can reach the app via https://immich.xxx.xxx.xxx.xxx.sslip.io.

jata1 · Oct 9th 2024

Thanks votdev

Good to know it is likely a DNS issue. I run opnsense firewall/router and this is usually working perfectly for DNS but I will investigate.

I can currently resolve omv.lan and omv.local to the correct IP address so this is fine. The issue is with dns resolution of subdomain.omv.lan so I will see if I can fix this so subdomains resolve to same ip as omv.lan

jata1 · Oct 9th 2024

Just an update to this thread as I have kubernetes working (thanks for your help votdev)

It did require me to make a tricky/quirky change in my opnsense/unbound settings and add a wildcard host override.

Now anything.omv.lan resoles to the ip of omv.lan as configured in the override.

I personally do not like this solution so I will see if I can find a better way in opnsense/unbound.

help to access my first kubernetes container

jata1 Oct 10th 2024

Participate now!