openmediavault-compose sample compose files

ryecoaaron · Jul 13th 2022

I was thinking that I could offer an option on the openmediavault-compose plugin to create example compose files.

I envision a dropdown that is populated with sample compose file names from a file on github. The user would select the item (nginx for example) and it would download the sample compose file and create the entry in the Files list of the compose plugin (disabled of course). The user would then edit the sample compose file to change the volume paths to a path on their system.

Does that sound like a good use of time? If yes, would people be interested in contributing compose files? The contribution could be done via post, PM, or github pull request.

HotBlack · Jul 13th 2022

I think that would be a truely awesome idea! I would certainly be happy to contribute some container templates.

Would it make senese to have some sort of "variable subsitution" in the compose files?
I am thinking something like:

1) Another tab in plugin which lets you define variables and strings they resolve to (e.g. %PID is replaced by '1003'), a user would fill these out the way they want them.

2) You download a template, and the templates which follow some sort of OMV standard, would use the same set of standard variables.

3) When you run docker up, the file is preprocessed and the varibales are swapped with strings or an error is thrown if a variable is used but not defined.

The idea here is that it would save users havng to explcitly define things like the same shared volume full of movies refered to in many containers and hopefully make it easier for users to get basic conatiners up and working.

ryecoaaron · Jul 13th 2022

Quote from HotBlack

Would it make senese to have some sort of "variable subsitution" in the compose files?

While I like the idea and I think the variables that need to be changed should be obvious, there would be an unknown number of variables in the compose file. Making the plugin that dynamic is really not feasible.

Zoki · Jul 13th 2022

How about using the built in capability of docker-compose to use a .env file.

Not a string substitution, no error checking, but will be enough for most purposes.

ryecoaaron · Jul 13th 2022

Quote from Zoki

How about using the built in capability of docker-compose to use a .env file.
Not a string substitution, no error checking, but will be enough for most purposes.

Do you think the plugin should let you edit the .env then? I would have to change the plugin to put each compose file in a sub-folder though.

Zoki · Jul 13th 2022

Yes and putting each docker-compose.yml into its own directory will allow for docker-compose local directories to be mounted

This is how I use docker-compose:

Code

.docker-apps
├── grafana
│   ├── config
│   ├── data
│   └── docker-compose.yml
├── influxdb-and-tools
│   ├── docker-compose.yml
│   ├── influxdb
│   └── telegraf
├── jellyfin
│   ├── config
│   └── docker-compose.yml
├── mosquitto
│   ├── config
│   ├── data
│   ├── docker-compose.yml
│   └── log
├── nextcloud
│   ├── docker-compose.yml
│   ├── mariadb
│   └── nextcloud
├── seafile -> /media/279518ea-0b44-479f-952f-d549dca76193/seafile-docker
├── syncthing
│   ├── config
│   └── docker-compose.yml
├── telegraf_mqtt
│   ├── docker-compose.yml
│   └── telegraf.conf
├── traefik
│   ├── acme.json
│   ├── conf.d
│   └── docker-compose.yml
├── urbackup
│   ├── backups -> /srv/zfs/pool1/urbackup
│   ├── docker-compose.yml
│   └── urbackup
└── vaultwarden
    ├── data
    └── docker-compose.yml

Display More

Not show are the .env files.

ryecoaaron · Jul 18th 2022

With 6.0.4 of the compose plugin (in the repo now), it puts each compose file in a sub-directory with the same name as the file and creates and environment file with the same name and extension of .env.

aaron@omv6dev:/srv/dev-disk-by-uuid-f4986fb7-838b-41ab-bc30-cfd22e29a4d1/composeFiles$ find .

.

./nginx

./nginx/nginx.yml

./nginx/nginx.env

Zoki · Jul 21st 2022

Quote from ryecoaaron

With 6.0.4 of the compose plugin (in the repo now), it puts each compose file in a sub-directory with the same name as the file and creates and environment file with the same name and extension of .env.

aaron@omv6dev:/srv/dev-disk-by-uuid-f4986fb7-838b-41ab-bc30-cfd22e29a4d1/composeFiles$ find .
.
./nginx
./nginx/nginx.yml
./nginx/nginx.env

Display More

Do you want to make me switch from the console to your plugin?

ryecoaaron · Jul 21st 2022

Quote from Zoki

Do you want to make me switch from the console to your plugin?

I haven't but if you did, I'm sure you would have good suggestions : ) Like the env file...

ryecoaaron · Oct 7th 2022

Does anyone want to contribute sample compose files for this plugin? Just post them here.

gderf · Oct 7th 2022

Here are all of mine.

compose-files.zip

ryecoaaron · Oct 9th 2022

openmediavault-compose 6.1 is in the repo with the sample compose file feature. If anyone wants to add a new compose file or tweak an existing compose file, file an issue or merge request against https://github.com/OpenMediaVault-Plugin-Developers/packages. If you add a new file, please run the update.sh script to properly update the list.json file needed for the plugin to know which compose files exist.

Soma · Dec 4th 2022

So, I finally decided to drop the CLI and test this.

Simple and effective.

One question though: wasn't the plugin capable of pulling from portainer the info from the running STACKs and paste it into a compose yml?
I think I remember reading about this but maybe I'm mixing threads/subjects.

Or maybe it was a post from Zoki (at the time) that refered that there was a command that could do this.

And I think this would be a major uplifter to the plugin, if it's doable.

macom · Dec 4th 2022

Do you mean this?

Docker Hub

Soma · Dec 4th 2022

Quote from macom

Do you mean this?
https://hub.docker.com/r/red5d/docker-autocompose

Something like that, yes.

Wasn't the plugin already capable of pulling that info?

ryecoaaron · Dec 4th 2022

Quote from Soma

Wasn't the plugin already capable of pulling that info?

Not yet.

ryecoaaron · Dec 6th 2022

Quote from Soma

Wasn't the plugin already capable of pulling that info?

The autocompose feature is added in 6.2 in the repo now. Just a warning, autocompose doesn't always make the best compose file.

BernH · Dec 6th 2022

Quote from ryecoaaron

The autocompose feature is added in 6.2 in the repo now. Just a warning, autocompose doesn't always make the best compose file.

I use the autocompose docker. I agree the files are not the best. Anytime I have had to use them I have to clean them up first, but they do serve as a quick way to re-deploy without having to read documentation and go through trial and error again with some containers.

I'll have to have a look at the new feature.

Soma · Dec 7th 2022

Quote from ryecoaaron

Just a warning, autocompose doesn't always make the best compose file.

So, I'm trying some things now on this one:

I have a piwigo STACK running via docker-compose (includes piwigo and mariadb) and tried to use the autocompose function.

What I can tell upfront is this doesn't work with STACKs (a limitation of autocompose itself, NOT the plugin, for sure) since it only allows to select 1 container.

After trying to create a YML of only the piwigo part, it errored:

Code

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 piwigo' with exit code '1': 

OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 piwigo' with exit code '1':  in /usr/share/php/openmediavault/system/process.inc:217
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/compose.inc(313): OMV\System\Process->execute(Array, 1)
#1 [internal function]: OMVRpcServiceCompose->doAutocompose(Array, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('doAutocompose', Array, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Compose', 'doAutocompose', Array, Array, 1)
#5 {main}

So, I moved on and tried with Portainer since it's a single container and NOT a STACK:

Same error:

Code

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 portainer' with exit code '1': 

OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 portainer' with exit code '1':  in /usr/share/php/openmediavault/system/process.inc:217
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/compose.inc(313): OMV\System\Process->execute(Array, 1)
#1 [internal function]: OMVRpcServiceCompose->doAutocompose(Array, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('doAutocompose', Array, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Compose', 'doAutocompose', Array, Array, 1)
#5 {main}

RIG:

OS drive 16Gb ext4

DATA 2x 20Gb RAID0 BTRFS

Soma · Dec 7th 2022

Quote from ryecoaaron

Just a warning, autocompose doesn't always make the best compose file.

So, I'm trying some things now on this one:

I have a piwigo STACK running via docker-compose (includes piwigo and mariadb) and tried to use the autocompose function.

What I can tell upfront is this doesn't work with STACKs (a limitation of autocompose itself, NOT the plugin, for sure) since it only allows to select 1 container.

After trying to create a YML of only the piwigo part, it errored:

Code

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 piwigo' with exit code '1': 

OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 piwigo' with exit code '1':  in /usr/share/php/openmediavault/system/process.inc:217
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/compose.inc(313): OMV\System\Process->execute(Array, 1)
#1 [internal function]: OMVRpcServiceCompose->doAutocompose(Array, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('doAutocompose', Array, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Compose', 'doAutocompose', Array, Array, 1)
#5 {main}

So, I moved on and tried with Portainer since it's a single container and NOT a STACK:

Same error:

Code

Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 portainer' with exit code '1': 

OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; export LANGUAGE=; python3 /usr/bin/autocompose.py -v 3 portainer' with exit code '1':  in /usr/share/php/openmediavault/system/process.inc:217
Stack trace:
#0 /usr/share/openmediavault/engined/rpc/compose.inc(313): OMV\System\Process->execute(Array, 1)
#1 [internal function]: OMVRpcServiceCompose->doAutocompose(Array, Array)
#2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('doAutocompose', Array, Array)
#4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Compose', 'doAutocompose', Array, Array, 1)
#5 {main}

RIG:

OS drive 16Gb ext4

DATA 2x 20Gb RAID0 BTRFS

Participate now!