Problem with docker-client on nfs-mounted filesystem

  • Hello everyone,


    OMV-NAS, 7.0-32 (sandworm)

    Various NFS folders (FS is Ext4)

    5x Raspberry in cluster -> Docker Swarm Cluster, boots over the network from the OMV

    Root-FS is Mounted with NFS Verse 4.2



    I'm facing a challenging issue with my Docker setup on a Raspberry Pi cluster running RaspOS 64-bit. The root filesystem of my nodes is mounted via NFS from OMV.
    I've encountered a persistent problem when trying to pull Docker images on the Raspi, specifically frooodle/s-pdf:latest. The error message is:

    Code
    failed to register layer: lsetxattr security.capability /usr/libexec/gstreamer-1.0/gst-ptp-helper: operation not supported

    This error seems to be related to setting extended attributes on files within the Docker image layers, which apparently is not supported by my NFS setup.
    (https://github.com/moby/moby/issues/45417)
    Here's a brief overview of what I've tried and the setup:

    • The cluster node's root FS is on NFS on OMV. Initially, it was NFSv3 (because TFTP needs v3, but it is another NFS-directory), but I've successfully migrated to NFSv4.2 to possibly leverage better support for extended attributes, as suggested by newer Linux kernels (5.9+).
      (https://www.phoronix.com/news/…5.9-NFS-Server-User-Xattr)
    • Despite changing to NFSv4.2, the issue persisted. Reading through various forums and documentation, I found that Docker's overlay2 storage driver might be incompatible with NFS for certain operations like setting extended attributes.
    • I installed fuse-overlayfs on the problematic node and configured Docker to use fuse-overlayfs as the storage driver.
      (https://docs.docker.com/storag…er/select-storage-driver/) This change was successful, and now docker info confirms that Storage Driver: fuse-overlayfs is active. However, the initial problem with pulling the Docker image remains unchanged.
    • docker pull frooodle/s-pdf:latest works without issues on an other Raspi bootet from USB.

    I'm seeking advice on how to resolve this issue or any insights into whether my current approach with fuse-overlayfs and NFSv4.2 is viable. Is there a configuration step I'm missing, or is there a better way to handle Docker images on an NFS-backed filesystem on OMV?

    I appreciate any help or pointers you can provide. Thank you in advance!

  • ryecoaaron

    Hat das Thema freigeschaltet.
  • Can you explain me how to do it step-by-step? i'm newbie for this kind of things.

    im trying to pull docker onlyoffice and facing same issue with output:

    failed to register layer: lsetxattr security.capability /usr/lib/aarch64-linux-gnu/gstreamer1.0/gstreamer-1.0/gst-ptp-helper: operation not supported

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!