Issues with installing docker in a SnapRAID world

  • I am running OMV 5 under proxmox. I have attempted to install docker under omv-extra, but when I attempt to change docker storage from "/var/lib/docker" to "/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Docker", docker-ce fails to install. I am trying to move docker storage to my SnapRAID pool. I have the permissions setup with Administrator (RW), Users (RW) and Others (RO) for the Docker shared folder. I always seem to screw up permissions and I am hoping this is just a problem with permissions.

  • What are the filesystem permissions and ownership of that directory (not what you see or set for the shared folder in OMV).


    Also, you may run into problems with SnapRaid throwing a warning flood at you if you run a sync without excluding some, most, or possibly all of that shared folder.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 16GB ECC RAM.

  • What are the filesystem permissions and ownership of that directory (not what you see or set for the shared folder in OMV).


    Also, you may run into problems with SnapRaid throwing a warning flood at you if you run a sync without excluding some, most, or possibly all of that shared folder.

    The permissions is:


    drwxrwsrwx 3 root users 4096 Sep 9 19:55 Docker


    I changed the permission to Admistrator (RWX), Users (RWX) and Other (RWX) and still no joy. Of course, I am able to install at /var/lib/docker, and I thought maybe I could do a ln from /var/lib.docker/containers to a shared folder, and create a shared folder for configs, but not ready to try that one.


    I am just started trying my hand with SnapRAID after I found out that I could not make use of zfs. I have not tried to do a snapraid sync, because although I have created the shared folder, as of this moment I have not added any content this time. So could you explain how to exclude the shared folder (or shared folders).

  • Uninstall docker from OMV GUI. Reboot


    From CLI, delete the folder you created for docker:

    sudo rm -rf /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Docker


    On OMV GUI extras: change the path to what you want the docker root to be (in your case: /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Docker )

    Install Docker on the button.

  • So I don't need to create a shared folder? Actually, I did follow up on your instructions and still no joy! I tried to copy the error message, but was not able to do so... But there is still an issue with install docker-ce.

  • When you install docker, the folder that is on the "root path" is created by OMV with the proper permissions.

    You don't make a share for it.


    But if you have any error with docker-ce, than something more might be wrong.


    Uninstall again docker, reboot and than run sudo systemctl status docker.service

  • I removed docker from the GUI, and checked to see if docker.service was running, which it wasn't. Where is the script to install docker from the GUI? Maybe I can use shellcheck and see how the installation script works and see where it fails..

  • I only use the "drop-down" menu on the "System->OMV-Extras->Docker" to install it.

    Prior to install, it's only a matter of setting the "Docker Storage" to the place you want it to be on a already mounted disk. (On my case I just wrote "/srv/dev-disk-by-label-sd_configs/@docker")

    Everything is installed after that to the specified folder.


    Maybe you can try doing a "apt clean" on the "System->OMV-Extras->Updates" drop-down menu.


    And/Or, maybe reinstall "OMV-Extras" on the "System->Plugins" menu.



    All the files for the Extras is here

  • Thanks for the pointers to the Extras files. I am wondering if my use of trying to make use of the pool directory might be causing the issues.


    In attempting to create a docker storage, I created a directory in pool-01. I am thinking that maybe I should just create a directory in one of the uid devices, because when synced, the docker storage will be placed in the pool-01. Just a thought...

  • So the problem as been solved! My problem was attempting to create the docker storage in the pool as opposed to point to one of the drives, creating the docker storage directory, and then syncing to the snapraid pool. Thanks to Soma and gderf for the help!

  • It has been a couple of weeks, but I have come on another issue. Or maybe it is not an issue and I am somewhat confused. I make use of Portainer to handle all of my docker containers (most of the time... Sometimes I need to just start one up on the command line).


    Code
    root@nas-01:/srv# ls -l
    total 40
    drwxr-xr-x 4 root root 4096 Oct 2 23:37 1abd74ac-84b5-4f06-a458-d5d87ecd6e1e
    drwxr-xr-x 5 root root 4096 Oct 3 08:57 dev-disk-by-label-media
    drwxr-xr-x 10 root root 4096 Oct 6 12:55 dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19
    drwxr-xr-x 4 root root 4096 Oct 2 23:37 dev-disk-by-uuid-2e164b03-9001-464b-bdb7-fef2a0b05ff1
    drwxr-xr-x 3 root root 4096 Sep 10 00:50 dev-disk-by-uuid-a3f78985-a6ea-43b3-8753-895c0e249b15
    drwxr-xr-x 8 root root 4096 Oct 2 23:37 dev-disk-by-uuid-cbf8c644-5871-4932-ab87-382b311cb786
    drwxr-xr-x 5 root root 4096 Oct 2 23:37 dev-disk-by-uuid-d02056de-d1d8-4046-bb52-2884b2847bfb

    When I try to for instance, bind (host) /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Configs to (container) /config, I tend to have some issue. On the other hand, if I bind /dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/Configs to /config, life is groovy. Am I suppose to use the latter method or did make a mistake in my configuration of snapraid? I would think that I should have to bind from /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e and not be concerned with where the directory is located on the drives. I find myself having to constantly ssh into the host to get the correct directory to Configs (ie /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/Configs) as opposed to /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Configs.


    Of course, this could just be my misinterpretation and I am not understanding correctly. Any pointers would be greatly appreciated!

  • Do not use shared folders or mergerfs pools in bind mounts.


    SnapRAID has nothing to do with this.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 16GB ECC RAM.

  • Do not use shared folders or mergerfs pools in bind mounts.


    SnapRAID has nothing to do with this.

    I might have misused the term incorrectly. I am still making my way in understanding how docker works. I guess I should have used volume???

  • So I have removed all of my shared folders because I wanted to simplify the structure. I am still perplexed by the action of using the path. For instance (also in the above posting), I have a directory in the pool called appdata (/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata), appdata is actually located at /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata.


    When I am spinning up a container, I try to use the following:


    Code
    /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata -> /config

    I start the container, but even though it says that it running, I am not able to access the container. If I change the container with the following:

    Code
    /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata -> /config

    re-deploy the container, I am able to access the container and life is wonderful.


    I have checked the permissions and everything is go, so I wondering why I can't use the shorter name?

  • It is not a good idea to use a mergerfs pool or a shared folder as a bind mount for a docker container's /config.


    One reason is that there can be race conditions at system startup such that the container is trying to start before the mergerfs pool is operational. In this case the container just won't work.


    Another thing that can go wrong is that all the files that are in the container /config can be spread out across more than one drive because of the pooling. If any one of those disks in the pool later winds up becoming unavailable, your container that was working stops working.


    If you need to use a shorter name for a directory, try using a symlink for it.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 16GB ECC RAM.

  • It is not a good idea to use a mergerfs pool or a shared folder as a bind mount for a docker container's /config.


    One reason is that there can be race conditions at system startup such that the container is trying to start before the mergerfs pool is operational. In this case the container just won't work.


    Another thing that can go wrong is that all the files that are in the container /config can be spread out across more than one drive because of the pooling. If any one of those disks in the pool later winds up becoming unavailable, your container that was working stops working.


    If you need to use a shorter name for a directory, try using a symlink for it.

    I was waiting for that statement from someone experienced since this thread was started.


    I had the same problem a few months ago. In the end I installed a small SSD that I had in a drawer dedicated to docker.

    The best thanks to the help provided is to report what your solution was. The next one will thank you :thumbup:

  • In the end I installed a small SSD that I had in a drawer dedicated to docker.

    I did the same thing. I had a 256GB 2.5in SSD lying around that I had no other use for. So I dedicated it to my docker stuff, my Plex metadatabase, and miscellaneous other needs.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 16GB ECC RAM.

  • I have tried to use a symlink, but with mixed results. Sometime, I can (for instance) use as my host link, /srv/appdata/foo (which is symlinked to/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata/) --> /config and it works just fine. On the other hand, on some containers, I have to do the full path /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata/bar --> /config or even /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata/bar --> /config. Each container acts a little different!


    Interestingly enough, I have never created a mergerfs folder. When looking at the tutorials on using snapraid, I never saw an example which use mergerfs. I did read about exempting directories in the /etc/snapraid.conf file (which I didn't know about) to protect the appdata folder.

  • SnapRaid and mergerfs are different things and do not interoperate with each other. So if you aren't using mergerfs you don't need to pay attention to suggestions about its use.


    Some programs that can be run in dockers have /config container side directories that will cause a lot of grief for SnapRaid unless they are excluded. One example is Plex's /config. Another is Smokeping. And there are many more. Rather than discover these one by one and exclude them one by one, I found it easier to put these all in one folder, confined to one drive, and exclude that folder. This leaves them all unprotected by SnapRaid, but I can live with that.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 16GB ECC RAM.

  • SnapRaid and mergerfs are different things and do not interoperate with each other. So if you aren't using mergerfs you don't need to pay attention to suggestions about its use.


    Some programs that can be run in dockers have /config container side directories that will cause a lot of grief for SnapRaid unless they are excluded. One example is Plex's /config. Another is Smokeping. And there are many more. Rather than discover these one by one and exclude them one by one, I found it easier to put these all in one folder, confined to one drive, and exclude that folder. This leaves them all unprotected by SnapRaid, but I can live with that.

    So for instance, my appdata currently resides in /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata. I need to exclude that folder in /etc/snapraid.conf and also directly reference the full path when I am configuring a container (ie: no symlinks).

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!