Compose plugin for bakups only

  • Hello,

    I have been using OMV since V4. I have it currently running on 3 devices - VM, x64, and ARM devices. All devices run docker along with several containers managed initially by the compose plugin. which in general provides kind of convenient GUI but the feature which stands out the best is the backing up mechanism. Unfortunately i spotted a few situations where using the compose plugin makes the things a bit harder for me. I am used to portainer and the easier way to manage networks, to attached a network to a running container, connecting remotely to a docker API socket etc. it is so easy an intuitive so i moved back to using portainer but try to find a way to keep the back up approach by the compose plugin.

    I have the following main points:

    1. I use btrfs raid 1 so i have to manually modify daemon.json which gets sometimes overwritten by the compose plugin because of any reason

    2. Additionally I bind the docker folder and compose/container/persistent data folder (or how you wanna call it) into an external ssd in order to protect SD card. This is also done pretty easy by manually editing the daemon.json

    3. Why should the compose backup feature affect only the compose files which are created within the plugin? Is there a technical restriction? It makes much more sense to backup everything. Docked for example provides editing possibilities for all containers in the corresponding folder


    Just want to share my feedback. I turned out the uninstalling the compose plugin and install everything manually works much better for me.


    Regards

  • chente

    Hat das Thema freigeschaltet.
    • Offizieller Beitrag

    The compose plugin backup utility is a script that reads the paths to backup from the compose files configured WITHIN the plugin. These files are in the OMV database, this is how plugins work in general, storing the information in this database.

    So backing up containers that have been deployed outside of the plugin would require additional work that I think ryecoaaron won't be willing to do. Anyway, it is ryecoaaron who should answer this question, but you will have to wait for him to return, he is traveling this week.

    • Offizieller Beitrag

    backing up containers that have been deployed outside of the plugin would require additional work that I think ryecoaaron won't be willing to do.

    Correct.

    • Offizieller Beitrag

    I use btrfs raid 1 so i have to manually modify daemon.json which gets sometimes overwritten by the compose plugin because of any reason

    As the Docker storage field states, if you leave it blank, the plugin will not modify daemon.json.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hello, i have been playing for quite some time with the compose plugin and i think this is a great piece of software. I was able to setup my containers as a wanted working with btrfs. I like the scheduling feature to backup and update images/compose files. I am getting rid of the portainer that i used mainly for network management. I should improve my skills doing that within the compose files.


    Here are a hew spots i would like to address:

    - The Scheduler Prune option seems to be missing even though it is still described to be supported option according to the documentation. I think it is a nice add on to keep the system clean and start the pruning process as soon as the u[date process is complete

    - I use a common compose file to bring up two/multiple containers that have a few dependencies: for example i have unifi network application running along with mongo db. Both are set to reset policy set to "unless-stopped". After the backup job was completed i found out that one of the containers was stopped. The long was fine - no errors what so ever. How does the backup script behaves in that case?

    -- Stop all of the containers; back them up; start all of the containers again

    or

    -- Stop the first container; back it up; start it up again ;and proceed with the next one


    The documentation says that the containers will be started if their last state before launching the backup process was running. I believe my containers were both running.


    - Is there an option to expand the "containers" tab by adding up the networks currently assigned to each container. I am also tempted to mention that being able to attach/set networks manually will be an awesome for me even though it is probably not the scope of the plugin.


    Thank you very much for the effort you put on this great distro which is may nas goto options for years


    Regards

    Yordan

    • Offizieller Beitrag

    The Scheduler Prune option seems to be missing even though it is still described to be supported option according to the documentation. I think it is a nice add on to keep the system clean and start the pruning process as soon as the u[date process is complete

    This is available on the OMV 7.x version of the plugin.


    I use a common compose file to bring up two/multiple containers that have a few dependencies: for example i have unifi network application running along with mongo db. Both are set to reset policy set to "unless-stopped". After the backup job was completed i found out that one of the containers was stopped. The long was fine - no errors what so ever. How does the backup script behaves in that case?

    -- Stop all of the containers; back them up; start all of the containers again

    or

    -- Stop the first container; back it up; start it up again ;and proceed with the next one

    The backup executes a docker compose stop command. That should stop all containers in the compose file. Then it runs the backup. Then it executes a docker compose start command that should start all containers in the compose file.


    I recommend upgrading to OMV 7.x. There has been many improvements in the OMV 7.x version of the compose plugin.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Is there a migration guide somewhere published officially

    It really hasn't changed from this guide other than omv-update is now omv-upgrade.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I suppose i might have created a new topic however it is still related to the compose plugin


    I can not find a way to create a macvlan assigned to a vlan

    Here is the case :

    - I have a dual ethernet bond interface configured as br0

    - i have two vlans - 5, 10 and of course the default 1

    - i am able to create a macvlan in compose->networks with a parent interface set to br0 from the drop-down option which makes a perfect fit for the default vlan

    - I might create two additional openmediavault vlan interfaces (Network ->Interfaces) and use them as a parents to the additional macvlan corresponding to vlanid 5 and vlanid 10, however i do not want to expose my host (openmediavault) web server to the vlan networks. I should explicitly restrict that in the firewall which is another thing to remember

    - with portainer it is possible to create a macvlan with a parent interface set to br0.10 which will implicitly create the corresponding bridge dedicated to docker only and will be invisible to the host

    • Offizieller Beitrag

    - with portainer it is possible to create a macvlan with a parent interface set to br0.10 which will implicitly create the corresponding bridge dedicated to docker only and will be invisible to the host

    I will have to see if I can make that possible in the plugin.

  • what would be a very nice feature is to be able to see the currently assigned IP addresses to each container.

    What makes sense to me is adding a new column in the compose->containers

    And being able to attach/detach networks at runtime. It is a very comfortable when fine-tuning with inter-container communication without modifying the compose file

    • Offizieller Beitrag

    what would be a very nice feature is to be able to see the currently assigned IP addresses to each container.

    You can see that in the Stats tab when you click Inspect for a container.


    What makes sense to me is adding a new column in the compose->containers

    The Containers tab is populated from the output of docker container ls --all --no-trunc --format "json" which does not contain the ip address. Having the plugin look up the ip for each container would significantly slow the tab down.

    And being able to attach/detach networks at runtime

    Are you using docker network connect/disconnect to do that? What is your use case? The disconnect would be easy but the connect would a lot of work and I don't see many people using this feature.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hello,


    I have derived this behavior from Portainer again. Not sure which command it uses..


    But here is another question:


    I have shared the compose backup folder in order to be able to share it with duplicati. Duplicati will make a cloud backup to my docker backup folder. The problem i got is that the owner of the compose backup is the ROOT and i use admin account to read it from duplicati. However duplicati throws an access error every time compose writes to it. I guess that the permissions of the folder are being changed every time the root get access to it.


    Do not want to mess up much with the ACLs but would it be a reasonable solution to set the owner of the compose backup folder to admin as well? Do you see any issue with that later on?


    Thanks

    • Offizieller Beitrag

    I have shared the compose backup folder in order to be able to share it with duplicati. Duplicati will make a cloud backup to my docker backup folder. The problem i got is that the owner of the compose backup is the ROOT and i use admin account to read it from duplicati. However duplicati throws an access error every time compose writes to it. I guess that the permissions of the folder are being changed every time the root get access to it.


    Do not want to mess up much with the ACLs but would it be a reasonable solution to set the owner of the compose backup folder to admin as well? Do you see any issue with that later on?

    The plugin isn't changing permissions of the backup sharedfolder. The rsync it uses for backup is running as root but it should preserve the owner. Sharedfolders by default are owned by root:users. I would guess your admin user is not in the users group either? I don't see any problem setting the owner of the backup folder to admin though.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    I used the default admin user currently available in OMV and it seems to be part of USERS.

    I would not use the "admin" user for anything other than signing in to the web interface.


    What are the ownership and permissions on the directory?

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I have the duplicati container started with PGID and PUID values that correspond to the admin. This is what i got when ssh with "id admin". Do you mean it is better to use a newly created user?



    And I have additionally set Read/Write permissions to admin in ACL.


    The problem is with just one sub-folder in the backup folder. The other subfolders are ok.


    1. I have crated a new user that i explicitly provide Read/Write Permission for the backup folder. Then used the PGID and PUID of that user to recreate the duplicati container but i still got that access error when in tries to read it.


    2. Share the same folder with samba and logged in with the same user. Same problem: can not enter it


    3. Then i installed the "reset permission" plugin and reset the permissions to Admin-Rad/Write; User-Read/Write;Other - Read Only. Than i was able to access the problematic sub-folder. I have started the backup process again from the compose->schedule gui. And immediatelly after the backup was completed, the subfolder was again restricted.


    Just to mention that i have also activated Backup and Update options in the job

    4 Mal editiert, zuletzt von ynikolov () aus folgendem Grund: New information added

  • What i found out that each container has a folder 0. All folder with name 0 are actually locked after the backup proces. Could it be that docker locks that and the compose plugin/rsync copies the permissions as is already locked in the backup folder

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!