Issues with installing docker in a SnapRAID world

I am running OMV 5 under proxmox. I have attempted to install docker under omv-extra, but when I attempt to change docker storage from "/var/lib/docker" to "/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Docker", docker-ce fails to install. I am trying to move docker storage to my SnapRAID pool. I have the permissions setup with Administrator (RW), Users (RW) and Others (RO) for the Docker shared folder. I always seem to screw up permissions and I am hoping this is just a problem with permissions.

Zitat von gderf

What are the filesystem permissions and ownership of that directory (not what you see or set for the shared folder in OMV).

Also, you may run into problems with SnapRaid throwing a warning flood at you if you run a sync without excluding some, most, or possibly all of that shared folder.

The permissions is:

drwxrwsrwx 3 root users 4096 Sep 9 19:55 Docker

I changed the permission to Admistrator (RWX), Users (RWX) and Other (RWX) and still no joy. Of course, I am able to install at /var/lib/docker, and I thought maybe I could do a ln from /var/lib.docker/containers to a shared folder, and create a shared folder for configs, but not ready to try that one.

I am just started trying my hand with SnapRAID after I found out that I could not make use of zfs. I have not tried to do a snapraid sync, because although I have created the shared folder, as of this moment I have not added any content this time. So could you explain how to exclude the shared folder (or shared folders).

Zitat von Soma

Uninstall docker from OMV GUI. Reboot

From CLI, delete the folder you created for docker:

sudo rm -rf /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Docker

On OMV GUI extras: change the path to what you want the docker root to be (in your case: /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Docker )

Install Docker on the button.

Alles anzeigen

So I don't need to create a shared folder? Actually, I did follow up on your instructions and still no joy! I tried to copy the error message, but was not able to do so... But there is still an issue with install docker-ce.

Zitat von Soma

When you install docker, the folder that is on the "root path" is created by OMV with the proper permissions.

You don't make a share for it.

But if you have any error with docker-ce, than something more might be wrong.

Uninstall again docker, reboot and than run sudo systemctl status docker.service

Alles anzeigen

I removed docker from the GUI, and checked to see if docker.service was running, which it wasn't. Where is the script to install docker from the GUI? Maybe I can use shellcheck and see how the installation script works and see where it fails..

Zitat von Soma

I only use the "drop-down" menu on the "System->OMV-Extras->Docker" to install it.

Prior to install, it's only a matter of setting the "Docker Storage" to the place you want it to be on a already mounted disk. (On my case I just wrote "/srv/dev-disk-by-label-sd_configs/@docker")

Everything is installed after that to the specified folder.

Maybe you can try doing a "apt clean" on the "System->OMV-Extras->Updates" drop-down menu.

And/Or, maybe reinstall "OMV-Extras" on the "System->Plugins" menu.

All the files for the Extras is here

Alles anzeigen

Thanks for the pointers to the Extras files. I am wondering if my use of trying to make use of the pool directory might be causing the issues.

Filesystem                                     1K-blocks    Used   Available Use% Mounted on
udev                                             4042316       0     4042316   0% /dev
tmpfs                                             814604   10436      804168   2% /run
/dev/sda1                                       50356760 3264060    44504952   7% /
tmpfs                                            4073004       0     4073004   0% /dev/shm
tmpfs                                               5120       0        5120   0% /run/lock
tmpfs                                            4073004       0     4073004   0% /sys/fs/cgroup
tmpfs                                            4073004       0     4073004   0% /tmp
/dev/sdb1                                     3844550452    1844  3844532224   1% /srv/dev-disk-by-uuid-a3f78985-a6ea-43b3-8753-895c0e249b15
/dev/sdf1                                     3844550452     156  3649183036   1% /srv/dev-disk-by-uuid-cbf8c644-5871-4932-ab87-382b311cb786
/dev/sde1                                     3844550452      60  3649183132   1% /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19
/dev/sdd1                                     3844550452      56  3649183136   1% /srv/dev-disk-by-uuid-d02056de-d1d8-4046-bb52-2884b2847bfb
/dev/sdc1                                     3844550452      56  3649183136   1% /srv/dev-disk-by-uuid-2e164b03-9001-464b-bdb7-fef2a0b05ff1
pool-01:1abd74ac-84b5-4f06-a458-d5d87ecd6e1e 15378201808     328 14596732440   1% /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e

In attempting to create a docker storage, I created a directory in pool-01. I am thinking that maybe I should just create a directory in one of the uid devices, because when synced, the docker storage will be placed in the pool-01. Just a thought...

So the problem as been solved! My problem was attempting to create the docker storage in the pool as opposed to point to one of the drives, creating the docker storage directory, and then syncing to the snapraid pool. Thanks to Soma and gderf for the help!

It has been a couple of weeks, but I have come on another issue. Or maybe it is not an issue and I am somewhat confused. I make use of Portainer to handle all of my docker containers (most of the time... Sometimes I need to just start one up on the command line).

root@nas-01:/srv# ls -l
total 40
drwxr-xr-x  4 root root    4096 Oct  2 23:37 1abd74ac-84b5-4f06-a458-d5d87ecd6e1e
drwxr-xr-x  5 root root    4096 Oct  3 08:57 dev-disk-by-label-media
drwxr-xr-x 10 root root    4096 Oct  6 12:55 dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19
drwxr-xr-x  4 root root    4096 Oct  2 23:37 dev-disk-by-uuid-2e164b03-9001-464b-bdb7-fef2a0b05ff1
drwxr-xr-x  3 root root    4096 Sep 10 00:50 dev-disk-by-uuid-a3f78985-a6ea-43b3-8753-895c0e249b15
drwxr-xr-x  8 root root    4096 Oct  2 23:37 dev-disk-by-uuid-cbf8c644-5871-4932-ab87-382b311cb786
drwxr-xr-x  5 root root    4096 Oct  2 23:37 dev-disk-by-uuid-d02056de-d1d8-4046-bb52-2884b2847bfb

root@nas-01:/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e# ls -la
total 81696
drwxr-xr-x   4 root root      4096 Oct  2 23:37 .
drwxr-xr-x  12 root root      4096 Sep 13 03:21 ..
-rw-------   1 root root      6144 Sep 13 11:02 aquota.group
-rw-------   1 root root      6144 Sep 13 11:02 aquota.user
drwxrwsrwx   2 root users     4096 Sep  8 05:43 Audiobooks
drwxrwsrwx  13 root users     4096 Oct  6 11:09 Configs
drwx--x--x  13 root users     4096 Oct  2 23:41 Docker
drwxrwsr-x   5 root users     4096 Oct  2 23:06 Downloads
drwxrwsr-x+  5 root users     4096 Oct  1 23:57 Ebooks
drwxrwxr-x   3 root users     4096 Sep 22 05:04 Home
drwxrwsr-x   2 root users     4096 Sep 10 00:47 ISO
drwx------   2 root root     16384 Sep  8 01:40 lost+found
drwxrwsr-x+  2 root users     4096 Sep  8 05:45 Movies
drwxrwsr-x+  2 root users     4096 Sep  8 05:46 Music
-rw-------   1 root root  83534371 Oct  2 23:37 snapraid.content
-rw-------   1 root root         0 Oct  6 00:00 snapraid.content.lock
drwxrwsr-x+  2 root users     4096 Sep  8 05:47 Software
drwxrwsr-x+  4 root users     4096 Oct  2 05:12 Torrents
drwxrwsr-x+  2 root users     4096 Sep  8 05:46 TV
drwxrwsrwx   4 root users     4096 Oct  2 05:17 Usenet

When I try to for instance, bind (host) /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Configs to (container) /config, I tend to have some issue. On the other hand, if I bind /dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/Configs to /config, life is groovy. Am I suppose to use the latter method or did make a mistake in my configuration of snapraid? I would think that I should have to bind from /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e and not be concerned with where the directory is located on the drives. I find myself having to constantly ssh into the host to get the correct directory to Configs (ie /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/Configs) as opposed to /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/Configs.

Of course, this could just be my misinterpretation and I am not understanding correctly. Any pointers would be greatly appreciated!

Zitat von gderf

Do not use shared folders or mergerfs pools in bind mounts.

SnapRAID has nothing to do with this.

I might have misused the term incorrectly. I am still making my way in understanding how docker works. I guess I should have used volume???

So I have removed all of my shared folders because I wanted to simplify the structure. I am still perplexed by the action of using the path. For instance (also in the above posting), I have a directory in the pool called appdata (/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata), appdata is actually located at /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata.

When I am spinning up a container, I try to use the following:

/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata -> /config

I start the container, but even though it says that it running, I am not able to access the container. If I change the container with the following:

/srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata -> /config

re-deploy the container, I am able to access the container and life is wonderful.

I have checked the permissions and everything is go, so I wondering why I can't use the shorter name?

Zitat von gderf

It is not a good idea to use a mergerfs pool or a shared folder as a bind mount for a docker container's /config.

One reason is that there can be race conditions at system startup such that the container is trying to start before the mergerfs pool is operational. In this case the container just won't work.

Another thing that can go wrong is that all the files that are in the container /config can be spread out across more than one drive because of the pooling. If any one of those disks in the pool later winds up becoming unavailable, your container that was working stops working.

If you need to use a shorter name for a directory, try using a symlink for it.

Alles anzeigen

I have tried to use a symlink, but with mixed results. Sometime, I can (for instance) use as my host link, /srv/appdata/foo (which is symlinked to/srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata/) --> /config and it works just fine. On the other hand, on some containers, I have to do the full path /srv/1abd74ac-84b5-4f06-a458-d5d87ecd6e1e/appdata/bar --> /config or even /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata/bar --> /config. Each container acts a little different!

Interestingly enough, I have never created a mergerfs folder. When looking at the tutorials on using snapraid, I never saw an example which use mergerfs. I did read about exempting directories in the /etc/snapraid.conf file (which I didn't know about) to protect the appdata folder.

Zitat von gderf

SnapRaid and mergerfs are different things and do not interoperate with each other. So if you aren't using mergerfs you don't need to pay attention to suggestions about its use.

Some programs that can be run in dockers have /config container side directories that will cause a lot of grief for SnapRaid unless they are excluded. One example is Plex's /config. Another is Smokeping. And there are many more. Rather than discover these one by one and exclude them one by one, I found it easier to put these all in one folder, confined to one drive, and exclude that folder. This leaves them all unprotected by SnapRaid, but I can live with that.

So for instance, my appdata currently resides in /srv/dev-disk-by-uuid-1d060b23-6aec-4855-85e7-c0cb380c0d19/appdata. I need to exclude that folder in /etc/snapraid.conf and also directly reference the full path when I am configuring a container (ie: no symlinks).