Kubernetes on OMV7

I'm currently working on a Kubernetes plugin for OMV7.

The plugin is intended to implement a single-node Kubernetes environment using SUSE K3s. The plugin comes with preinstalled Traefik, Cert-Manager and Kubernetes Dashboard.

The OMV UI contains list views for the most important resources, but it is never intended to be a replacement for other UIs such as Kubernetes Dashboard or SUSE Rancher.

It will introduce so-called recipes that are simply manifests or that are based on existing Helm charts. This should make it easier for less experienced kubernetes users to get started.

I am planning a new GitHub repository where anyone interested can contribute such recipes for all kinds of applications. The UI will then integrate these and thus simplify the installation of applications.

Here are some simple example:

Filebrowser

apiVersion: v1
kind: Namespace
metadata:
  name: filebrowser-app
---
apiVersion: "v1"
kind: "PersistentVolumeClaim"
metadata:
  name: "shared-folder-data"
  namespace: filebrowser-app
spec:
  storageClassName: shared-folder
  accessModes:
    - "ReadWriteMany"
  resources:
    requests:
      storage: "2Gi"
  volumeName: "shared-folder-data"
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: filebrowser
  namespace: filebrowser-app
spec:
  repo: https://utkuozdemir.org/helm-charts
  chart: filebrowser
  targetNamespace: filebrowser-app
  valuesContent: |-
    securityContext:
      runAsNonRoot: true
      runAsUser: 1001
      runAsGroup: 100
    rootDir:
      pvc:
        existingClaim: "shared-folder-data"
    config:
      baseURL: /filebrowser
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: filebrowser-websecure
  namespace: filebrowser-app
spec:
  entryPoints:
    - websecure
  routes:
    - match: PathPrefix(`/filebrowser`)
      kind: Rule
      services:
        - name: filebrowser
          port: 80
  tls:
    secretName: host-selfsigned-cert

WeTTY

apiVersion: v1
kind: Namespace
metadata:
  name: wetty-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/instance: wetty
    app.kubernetes.io/name: wetty
  name: wetty
  namespace: wetty-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/instance: wetty
      app.kubernetes.io/name: wetty
  template:
    metadata:
      labels:
        app.kubernetes.io/instance: wetty
        app.kubernetes.io/name: wetty
    spec:
      containers:
      - image: docker.io/wettyoss/wetty:latest
        imagePullPolicy: IfNotPresent
        name: wetty
        ports:
        - containerPort: 3000
        securityContext:
          runAsNonRoot: true
          runAsUser: 65534
          runAsGroup: 65534
        args:
        - "--port 3000"
        - "--base /wetty"
        - "--force-ssh"
        - "--ssh-port 22"
        - "--ssh-host=<HOSTNAME>"
---
apiVersion: v1
kind: Service
metadata:
  name: wetty
  namespace: wetty-app
  labels:
    app.kubernetes.io/instance: wetty
    app.kubernetes.io/name: wetty
spec:
  type: ClusterIP
  ports:
  - name: wetty
    port: 3000
    protocol: TCP
    targetPort: 3000
  selector:
    app.kubernetes.io/instance: wetty
    app.kubernetes.io/name: wetty
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: wetty-websecure
  namespace: wetty-app
spec:
  entryPoints:
  - websecure
  routes:
  - match: PathPrefix(`/wetty`)
    kind: Rule
    services:
    - name: wetty
      port: 3000
  tls:
    secretName: host-selfsigned-cert

You can watch a video on the project Youtube channel to get a first impression about the plugin.

Zitat von lorenzo95

How would k3s be updated?

If necessary, k3s is reinstalled if the file /var/lib/openmediavault/upgrade_k3s exist. I have to say i have not tested that and i count on contributions from the community to improve the plugin.

Zitat von lorenzo95

Also, is k3s pretty default so I could use networkpolicy?

See https://docs.k3s.io/networking#network-policy-controller and https://www.suse.com/c/rancher_blog/k3s-network-policy/

Zitat von andrzejls

I just upgraded to OMV7 from 6 and tried to install this but there is no K3s plugin, only "openmediavault-k8s 7.0-1" which fail to install. Am I missing or doing sometging wrong?

There is openmediavault-k8s 7.0-2 out now which should fix that. As workaround simply run wget -O - https://get.k3s.io | INSTALL_K3S_SKIP_ENABLE=true sh - from CLI.

Zitat von JokerGermany

Would be nice to be able to change the data location used for k3s in the gui...

Why?

Zitat von JokerGermany

Hmm, i am not a docker or k8 expoert, but i try to make an recipe.

I change this docker-compose.yaml ( https://github.com/grocy/grocy…b/main/docker-compose.yml ) with kompose to k8 files.

Then I made one file and added a namespace:

YAML

apiVersion: v1
kind: Namespace
metadata:
  name: grocy
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  namespace: grocy
  labels:
    io.kompose.service: app-db
  name: app-db
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: grocy
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.32.0 (765fde254)
  labels:
    io.kompose.service: backend
  name: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: backend
  strategy:
    type: Recreate
  template:
    metadata:
      annotations:
        kompose.cmd: ./kompose convert
        kompose.version: 1.32.0 (765fde254)
      labels:
        io.kompose.network/k8-default: "true"
        io.kompose.service: backend
    spec:
      containers:
        - env:
            - name: GROCY_CULTURE
              valueFrom:
                configMapKeyRef:
                  key: GROCY_CULTURE
                  name: grocy-env
            - name: GROCY_MODE
              valueFrom:
                configMapKeyRef:
                  key: GROCY_MODE
                  name: grocy-env
            - name: MAX_UPLOAD
              valueFrom:
                configMapKeyRef:
                  key: MAX_UPLOAD
                  name: grocy-env
            - name: PHP_MAX_FILE_UPLOAD
              valueFrom:
                configMapKeyRef:
                  key: PHP_MAX_FILE_UPLOAD
                  name: grocy-env
            - name: PHP_MAX_POST
              valueFrom:
                configMapKeyRef:
                  key: PHP_MAX_POST
                  name: grocy-env
            - name: PHP_MEMORY_LIMIT
              valueFrom:
                configMapKeyRef:
                  key: PHP_MEMORY_LIMIT
                  name: grocy-env
          image: grocy/backend:v4.0.3
          name: backend
          ports:
            - containerPort: 9000
              protocol: TCP
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - mountPath: /var/www/data
              name: app-db
            - mountPath: /tmp
              name: backend-tmpfs0
            - mountPath: /var/www/data/viewcache
              name: backend-tmpfs1
      restartPolicy: Always
      volumes:
        - name: app-db
          persistentVolumeClaim:
            claimName: app-db
        - emptyDir:
            medium: Memory
          name: backend-tmpfs0
        - emptyDir:
            medium: Memory
          name: backend-tmpfs1
---
apiVersion: v1
kind: Service
metadata:
  namespace: grocy
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.32.0 (765fde254)
  labels:
    io.kompose.service: backend
  name: backend
spec:
  ports:
    - name: "9000"
      port: 9000
      targetPort: 9000
  selector:
    io.kompose.service: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: grocy
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.32.0 (765fde254)
  labels:
    io.kompose.service: frontend
  name: frontend
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: frontend
  template:
    metadata:
      annotations:
        kompose.cmd: ./kompose convert
        kompose.version: 1.32.0 (765fde254)
      labels:
        io.kompose.network/k8-default: "true"
        io.kompose.service: frontend
    spec:
      containers:
        - image: grocy/frontend:v4.0.3
          name: frontend
          ports:
            - containerPort: 8081
              hostIP: 192.168.2.2
              hostPort: 8081
              protocol: TCP
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - mountPath: /tmp
              name: frontend-tmpfs0
      restartPolicy: Always
      volumes:
        - emptyDir:
            medium: Memory
          name: frontend-tmpfs0
---
apiVersion: v1
kind: Service
metadata:
  namespace: grocy
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.32.0 (765fde254)
  labels:
    io.kompose.service: frontend
  name: frontend
spec:
  ports:
    - name: "8081"
      port: 8081
      targetPort: 8081
  selector:
    io.kompose.service: frontend
---
apiVersion: v1
data:
  GROCY_CULTURE: de
  GROCY_MODE: production
  MAX_UPLOAD: 50M
  PHP_MAX_FILE_UPLOAD: "200"
  PHP_MAX_POST: 100M
  PHP_MEMORY_LIMIT: 512M
kind: ConfigMap
metadata:
  namespace: grocy
  labels:
    io.kompose.service: backend-grocy-env
  name: grocy-env

Alles anzeigen

Now I have two question:
How do I configure an external port for the frontend (8081)?

How do i give app-db a path?

Alles anzeigen

The load balancer is running at port 8080 and 8443. You can use them if your app supports subpaths. If not you need to create a NodePort service. In both cases a IngressRoute resource is necessary.

And for accessing the local storage, you need a PVC that requests the shared folder PV.

Zitat von ness1602

I guess if you want to move the data folder to RAID shared folder or somewhere else?

But relocating that data to a different file system than root will make it necessary that the k3s service is waiting for the file system to become ready before starting the service.

I considered this when developing the plugin, but then discarded it because it means too much work and problems and this data is not important if the root file system is broken. All important data is in the etcd snapshots which are on a configurable shared folder. Everything else can be downloaded again by the system.