Docker compose plugin - Substitution string for shared folders paths in compose files

I really like the "CHANGE_TO_COMPOSE_DATA_PATH" substitution string that you can write in the docker compose files. It makes storing the configuration for the containers very easy. And it makes the files more portable between systems.

But what i noticed is that my compose files are still not portable between systems because in almost every file i have multiple "/srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/[name of shared folder]" strings because the containers need to access some data on my shared folders. And if i switch systems or change my hard drive i have to replace them all with the uuid of another hard drive.

Would it be possible to introduce another substitution string like for example "CHANGE_TO_SHAREDFOLDER_PATH[name]" that gets auto replaced by the plugin with "/srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/[name]"? I don't know if using "[" and "]" in that string is a good idea, that is just an example.

Then it would be possible to copy the compose files between systems as long as you keep the shared folder names the same.

Zitat von ryecoaaron

The file isn't meant to be portable. The database entry would be portable if the second system was running OMV and the compose plugin. If you cut and paste out of the web interface between the two systems, you wouldn't have an issue. Otherwise, the symlink or global env variable is the way.

I am not copying the .yml files from system to system. I have a text file with all my compose configurations from where i paste the text into the files section of the compose plugin webinterface. But its very annoying to redo all the "/srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/" strings every time.

The solution with global env variable is easy to do. I will just use that.

I have 3x NanoPi R6C here. Each one with Armbian Bullseye and OMV (installed with your installScript). And i need to run the same Docker Containers on each of them but the M.2 drives have different UUID.

Why do you think i am not using OMV?

Zitat von ryecoaaron

Because you mentioned that you were copying the compose file from a txt file instead of from the compose plugin with the CHANGE_TO_COMPOSE_DATA_PATH in the file. If you just update the txt file with the CHANGE_TO_COMPOSE_DATA_PATH string or cut & paste between omv web interface, it would automatically update the path based on what you have in the settings tab. No need to do anything with env file or symlinks or anything else.

I have that CHANGE_TO_COMPOSE_DATA_PATH string in my text file, but only for container "/config" volumes. The containers also need access to other shared folders.

I guess if i put shared folders inside shared folders and use the top one as Compose Data path that would work, but i would prefer not to do that and keep things separate.

(BEFORE)

volumes:

- CHANGE_TO_COMPOSE_DATA_PATH/Containername:/config

- /srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/SharedFolder1/SubFolder1:/data/SubFolder1

- /srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/SharedFolder2/SubFolder2:/data/SubFolder2

(THAT WAS MY IDEA)

volumes:

- CHANGE_TO_COMPOSE_DATA_PATH/Containername:/config

- CHANGE_TO_SHAREDFOLDER_PATH[SharedFolder1]/SubFolder1:/data/SubFolder1

- CHANGE_TO_SHAREDFOLDER_PATH[SharedFolder2]/SubFolder2:/data/SubFolder2

(WHAT I USE NOW)

volumes:

- CHANGE_TO_COMPOSE_DATA_PATH/Containername:/config

- $DrivePath/SharedFolder1/SubFolder1:/data/SubFolder1

- $DrivePath/SharedFolder2/SubFolder2:/data/SubFolder2

with global env:

DrivePath=/srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

I'm done now with installing all the systems and the global environment variable solution worked without a problem.

But i still think it would be a useful feature for the docker compose plugin to provide more information about the local system automatically to the containers.

Only having the information about the location of one single shared folder with "CHANGE_TO_COMPOSE_DATA_PATH" is very limiting.

If not with substitution strings then with global environment variables that get automatically inserted when you activate an option in the plugin.

(example)

OMV_COMPOSE_PUID_(Name)=xxxx (only for Users that display in omv webinterface)

OMV_COMPOSE_PGID_(Name)=xxxx

OMV_COMPOSE_LOCALTIMEZONE=xxxx/xxxxx

OMV_COMPOSE_SHAREDFOLDERPATH_(Name1)=/srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/SharedFolder1

OMV_COMPOSE_SHAREDFOLDERPATH_(Name2)=/srv/dev-disk-by-uuid-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/SharedFolder2

etc...

Zitat von ryecoaaron

The global environment variables do that now. If you add your example variables to the global to the plugin and the just use ${OMV_COMPOSE_SHAREDFOLDERPATH_Name1} in any compose file where you want to use that folder, it will use it. Same with the other variables in your example. Not sure how that could get any better.

I am talking about the omv compose plugin automatically creating these variables from information on the local system and inserting them into global environment as an option. Not writing them by hand.

Or you could split "Data" into "Container Data" and "Container Configuration" with 2 different shared folders and 2 string substitutions. So its possible to use "Container Configuration" for all the "/config"-folders and "Container Data" for all the persistent container data folders or reference already existing shared folders for data. But that would bloat the plugin config page i guess.

Don't get me wrong. These are just ideas. I don't know if you still want to add any features to the compose plugin or if you consider it feature complete.

Zitat von ryecoaaron

I don't how or when or what would do this. I guess timezone would be easy but it is possible someone would want to use UTC in one container and their local timezone in another. So, creating this by default might cause people to have to remove it.

These variables would to need a have a special unique name so they don't conflict with anything else.

You would not overwrite any default variables like "TZ". There would be an "OMV_COMPOSE_LOCALTIMEZONE" variable and for the docker compose file you would write:

environment:

- TZ=$OMV_COMPOSE_LOCALTIMEZONE

or

volumes:

- $OMV_COMPOSE_SHAREDFOLDERPATH_Name1/SubFolder1:/data/Subfolder1

These names are just placeholders. Since OpenMediaVault environment variables already start with "OMV_" this would be confusing. They would need to start with some other string.

Zitat von ryecoaaron

I think people are already confused having a sharedfolder for compose files, a sharedfolder for container data, and a path for docker storage. Adding another shared folder for container config would be confusing I think. I just put data in a data subfolder and config in a config subfolder of the container data shared folder.

Yes that is true. Even more shared folders would make it more complicated for beginners.

Zitat von chente

You can try this. https://docs.docker.com/compos…file-v3/#extension-fields

Here is how to use it. https://www.smarthomebeginner.…ocker-compose-guide-2022/

Seems nice if you have to repeat the same section for a large number of containers. I need to read more about YAML anchors i guess.

Zitat von ryecoaaron

So, there would just pre-populate the global environment file. I still not sure what would maintain that list. I guess saltstack could but then it would get weird if people wanted to delete some/all them.

I don't know what salt can do and where its limitations are. But i think people would not be happy if it messes with their perfectly handcrafted global environment. It would need to be separate and only inserted during container creation.

Zitat von olduser

Sounds problematic. How is a data directory automatically discerned? Is $HOME valid?

No. Just a few variables with basic info like timezone or shared folder paths with custom names. Or else it could be a security risk.

Maybe more users can comment if it would be helpful to them to reference shared folders by name in compose files or if its not needed. All my omv machines are running fine without it and some copy&pasting, but it can't hurt to throw around some ideas.