SFTP Plugin and ZFS

  • So I have been banging my head on this for a while. Basically I have moved my entire 20TB ext4 pool to a backup server and then started the process of moving everything into a new 20TB ZFS Pool. Crazy thing is that after hours and hours of trying to figure out why 2 of my ZFS datasets disappear I uninstalled SFTP. Once I restarted the ZFS unmounted and mounted as normal. Apparently SFTP does something using tmpfs and it conflicts with the mounts that ZFS uses. Aside from just uninstalling the SFTP plugin are there any ideas on how to get this working or any alternatives. If alternatives are suggested they must be encrypted links.


    Thank You, ALL!

  • Apparently SFTP does something using tmpfs and it conflicts with the mounts that ZFS uses.

    Nope. It bind mounts (no tmpfs) to paths in order to limit access for users but that shouldn't affect zfs. I would run a second sftp server in a docker. There are a lot of them on docker hub.

    omv 5.5.9 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • if SFTP binds mounts then that makes sense. Here is what happens...


    Code
    zfs list
    NAME USED AVAIL REFER MOUNTPOINT
    StorPoolMmbr01 12.5T 2.79T 363K /StorPoolMmbr01
    StorPoolMmbr01/ApplicationServerFS 465G 2.79T 465G /StorPoolMmbr01/ApplicationServerFS
    StorPoolMmbr01/BackupDataFS 672G 2.79T 672G /StorPoolMmbr01/BackupDataFS
    StorPoolMmbr01/DownloadsFS 192G 2.79T 192G /StorPoolMmbr01/DownloadsFS
    StorPoolMmbr01/MediaAlbumsFS 386G 2.79T 386G /StorPoolMmbr01/MediaAlbumsFS
    StorPoolMmbr01/MediaCollectionFS 9.56T 2.79T 9.56T /StorPoolMmbr01/MediaCollectionFS
    StorPoolMmbr01/SoftwareFS 1.04T 2.79T 1.04T /StorPoolMmbr01/SoftwareFS
    StorPoolMmbr01/UserDataFS 197G 2.79T 197G /StorPoolMmbr01/UserDataFS


    I broke up everything into different datasets


    when I see the root file system this is what I see...

    As you can see StorPoolMmbr01 is the mount point of my ZFS pool.


    These are the folders under StorPoolMmbr01.


    Code
    drwxr-xr-x 4 root root 4 Sep 28 01:49 ApplicationServerFS
    drwxr-xr-x 7 root root 7 Sep 28 02:10 BackupDataFS
    drwxr-xr-x 5 root root 5 Sep 30 08:47 DownloadsFS
    drwxr-xr-x 4 root root 4 Sep 28 02:18 MediaAlbumsFS
    drwxr-xr-x 10 root root 10 Sep 28 02:19 MediaCollectionFS
    drwxr-xr-x 3 root root 3 Sep 28 02:19 SoftwareFS
    drwxr-xr-x 5 root root 5 Oct 1 08:06 UserDataFS

    The problem is that when the machine is restarted the folders underneath UserDataFS and MediaCollectionFS are never empty on startup. When ZFS tries to mount the volume and all the datasets it fails on those two. I think this is because SFTP somehow puts folders under those that I have to delete. If those folders are left there bound then that makes sense, and would cause ZFS to fail to mount UserDataFS and MediaCollectionFS on startup. How would I know if SFTP is cleaning up during system shutdown? Perhaps there is something I can do to try and workaround. I don't want to start complicating things by using docker containers.

  • How would I know if SFTP is cleaning up during system shutdown?

    The plugin creates bind mounts in /etc/fstab. systemd *should* do the right thing.



    I don't want to start complicating things by using docker containers.

    How does that complicate things? You create an sftp docker on a port with one folder shared with the container. It achieves the separation very well. I use sftp and will probably switch to docker soon.

    omv 5.5.9 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • The plugin creates bind mounts in /etc/fstab. systemd *should* do the right thing.


    How does that complicate things? You create an sftp docker on a port with one folder shared with the container. It achieves the separation very well. I use sftp and will probably switch to docker soon.

    You are correct. It SFTP created two entries in the fstab that point to the folders on the ZFS mount. Somehow ZFS does not like it because it will not mount folders on restart that SFTP put in the fstab. I will probably move over to docker I am just not very familiar with how docker works and when I have tried to use it in the past it was just a pain. I get the concept of containers but updating them was a PITA in the past.

  • updating them was a PITA in the past.

    Look up watchtower


    Also which docker image do you use for SFTP? I don't see it in the list.

    I haven't switched. What list are you looking at? I see no shortage of images here - https://hub.docker.com/search?q=sftp&type=image

    omv 5.5.9 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • do you know if those docker images offer the same features? i.e:
    -access to different shares for different users
    -easy to define (web UI?) users/shares ?

    Didn't try it, huh? I would imagine that some of the images can give different users access to different shares. The web ui would be OMV and portainer. I still haven't switched to using a container although I'm not using the plugin either.

    omv 5.5.9 usul | 64 bit | 5.4 proxmox kernel | omvextrasorg 5.3.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!