Appdata folders' owners and groups change when adding a compose file or rebooting

1st Official Post

    I recently setup OMV and everything seems to work except my appdata folders' owners and groups keep changing whenever I add a compose file or reboot.


    Before adding a compose file:

    After adding a compose file (in this case Plex, but it happens with any compose file):

    After reboot:

    When the owners and groups change it causes the applications to stop working because they can't access their respective databases anymore. Using chown/chgrp to change the owners/groups fixes it, until I add another compose file at which point they all get set back to root/root. Any ideas on what I messed up?

    OMV: 7.7.4-1 (Sandworm) | Kernel: Proxmox 6.14.0-2-pve

    Plugins: openmediavault-backup 7.1.4 | compose 7.4.4 | fail2ban 7.0.1 | flashmemory 7.0.1 | kernel 7.1.8 | nut 7.0.7-1 | omvextrasorg 7.0.2 | scripts 7.1 | sharerootfs 7.0-1 | zfs 7.1.3

  • chente

    Approved the thread.

    In Services | Compose | Settings | Compose Files


    What user and group do you have for the Owner of directories and files and Group of directories and files?

    --
    Google is your friend and Bob's your uncle!


    A backup strategy is worthless unless you have a verified to work by testing restore strategy.


    OMV AMD64 7.x on headless Chenbro NR12000 1U Intel Xeon CPU E3-1230 V2 @ 3.30GHz 32GB ECC RAM.

    OMV AMD64 8.x on headless Tyan Thunder SX GT86C-B5630 1U Server with Intel Xeon Silver 4110 CPU @ 2.10GHz & 32GB DDR4 ECC RAM.

    This might be the same problem I had! I had the directory below appdata directly in the docker compose file - not a good idea.


    E.g. for heimdal

    DoDon't
    volumes: - ${PATH_TO_APPDATA}/heimdall/config:/config volumes: - ${PATH_TO_APPDATA}/heimdall:/config



    -> my problem recently



    It is intended that the root directory for a new service is owned by root (or the user defined in the plugin). Whenever a new file/compose/service is added in the compose-files part, the saltstack runs and manages the directories again. Meaning creating if not created and changing ownership to root. After adding a new file the apply changes button comes a bit late. If you skip that one and click edit again somehow the apply changes doesn't come back. That's what I did additionally with the wrong volumes mapping.


    2nd. edit: I am 99.99% sure! Down the service. Create subdirectory. Move files (don't move compose.override.yml, service-name.yml, service-name.env). Edit compose volume path to subdirectory. Restart.

    • Official Post

    Why are you putting data in the compose file directory? Those directories are supposed to be more locked down since there is often passwords involved. The whole point of the CHANGE_TO_COMPOSE_DATA_PATH variable is to use the data shared folder for volume storage. The plugin doesn't change that ownership.

    omv 8.0.10-2 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.6 | compose 8.1.3 | cterm 8.0 | borgbackup 8.1.5 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post
    Quote from erbsenzaehler

    At least I was following the omv-extras example on how to Configure a container step by step (Jellyfin). The only thing I was missing when I adapted the heimdall service was the subdirectory for /config. Then it looked exactly the same way as his is looking now.

    That example uses CHANGE_TO_COMPOSE_DATA_PATH. So, there is no need to worry about the permissions of the compose file directory.

    omv 8.0.10-2 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.6 | compose 8.1.3 | cterm 8.0 | borgbackup 8.1.5 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    On the part Customize the compose file the /config directory get's put below the appdata path.


    Code
        volumes:
      - ${PATH_TO_APPDATA}/jellyfin/config:/config   # See Comment 4
      - CHANGE_TO_COMPOSE_DATA_PATH/media:/media   # See Comment 4
    Quote

    In the first line we are mapping the /config folder of the jellyfin container to a folder on our system. The /config folder is the one that contains the jellyfin configuration files, the database, users and passwords, plugins, etc. We want this folder to be located on a drive with access speed. So we map it to our /appdata folder that we have configured on a fast disk, ideal for managing a large database. A fast disk will allow Jellyfin to quickly read movie covers, etc. from the television and everything will be better.

    • Official Post
    Quote from erbsenzaehler

    the /config directory get's put below the appdata path.

    Well, I didn't write it but I disagree with that. My opinion though.

    omv 8.0.10-2 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.6 | compose 8.1.3 | cterm 8.0 | borgbackup 8.1.5 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    Quote from erbsenzaehler

    It is intended that the root directory for a new service is owned by root (or the user defined in the plugin). Whenever a new file/compose/service is added in the compose-files part, the saltstack runs and manages the directories again. Meaning creating if not created and changing ownership to root. After adding a new file the apply changes button comes a bit late. If you skip that one and click edit again somehow the apply changes doesn't come back. That's what I did additionally with the wrong volumes mapping.


    2nd. edit: I am 99.99% sure! Down the service. Create subdirectory. Move files (don't move compose.override.yml, service-name.yml, service-name.env). Edit compose volume path to subdirectory. Restart.

    Thanks, this sounds like what I did wrong. I'll look into moving the files around later this weekend:thumbup:



    Quote from ryecoaaron

    Why are you putting data in the compose file directory? Those directories are supposed to be more locked down since there is often passwords involved. The whole point of the CHANGE_TO_COMPOSE_DATA_PATH variable is to use the data shared folder for volume storage. The plugin doesn't change that ownership.

    I've never used a linux before, so I followed the omv-extras guide for getting started, and then I followed the omv-extras docker section for Jellyfin, which has you replace CHANGE_TO_COMPOSE_DATA_PATH with ${PATH_TO_APPDATA}/jellyfin/config. After Jellyfin was working I went to add Radarr, but the LinuxServer.io compose file said:

    Code
    volumes:
  - /path/to/radarr/data:/config

    I wasn't sure if they meant to do it the same way that Jellyfin was setup, so I clicked their 'click here for more info' link which listed examples:

    Code
    volumes:
  - /home/user/appdata/heimdall:/config

    So my smooth brain said, a-ha, appdata followed by the name of the application. After that I followed the same method for every container.

    OMV: 7.7.4-1 (Sandworm) | Kernel: Proxmox 6.14.0-2-pve

    Plugins: openmediavault-backup 7.1.4 | compose 7.4.4 | fail2ban 7.0.1 | flashmemory 7.0.1 | kernel 7.1.8 | nut 7.0.7-1 | omvextrasorg 7.0.2 | scripts 7.1 | sharerootfs 7.0-1 | zfs 7.1.3

    • Official Post
    Quote from ryecoaaron

    Why are you putting data in the compose file directory? Those directories are supposed to be more locked down since there is often passwords involved. The whole point of the CHANGE_TO_COMPOSE_DATA_PATH variable is to use the data shared folder for volume storage. The plugin doesn't change that ownership.

    I believe I should protect persistent data and compose files equally. Both contain sensitive data, passwords, etc.

    Quote from ryecoaaron

    Well, I didn't write it but I disagree with that. My opinion though.

    There are infinite ways to configure containers. The Docker in OMV document describes one that works and tries to be as educational as possible for beginners to understand. It also works for any hardware configuration.


    The CHANGE_TO_COMPOSE_DATA_PATH variable can be useful in some simple systems, but not all. The fact that there is only one variable is somewhat limiting. In basic systems with a single file system, it will work without problems, but if there are multiple file systems with data, the user would have to group them to use that variable in all containers, for example with mergerfs, and that would complicate things. Alternatively, it could be combined with other methods to complete the configurations. That's why I left it optional in the document, focusing instead on the use of global variables to define paths (or symbolic links, alternatively). The user is given the option to use this variable anyway, and its use and what it does are explained, of course.


    In my case, for example, and I'm sure it's the case for many users as well, documents, media files, and Docker and KVM files live on different file systems. And it doesn't seem reasonable to me to complicate my configuration to use that variable in all containers by grouping those file systems together if there are other, simpler methods.


    Grouping persistent data folders in the appdata folder seemed like the best way to keep things simple when I started using this plugin. I actually copied the idea from the system Zoki used, which he posted a long time ago on the forum. This system was a perfect fit for the plugin, so I did it that way, and it's worked for me for years without any issues. Therefore, I also wrote the wiki document this way.


    This has been mentioned previously in the forum, and your response was that you didn't want to complicate the plugin with additional variables to do the same thing this one already does. It would be easier to use this variable if there were at least three (at least in my case), but this isn't definitive; there may be other cases. Using global environment variables or symlinks is a universal solution.



    Quote from wkruspe

    this sounds like what I did wrong.

    The documentation explains how to do this correctly in several places. I've added a warning about this possible permissions change if done incorrectly. I hope this helps others.

    omv7:docker_in_omv [omv-extras.org]


    Quote

    Make sure to create subfolders within each appdata folder for each container folder.

    Don't do this: - /srv/dev-disk-by-uuid-9d43cda9-20e5-474f-b38b-6b2b6c03211a/appdata/jellyfin:/config

    If you do this, the persistent data in the config folder will be mixed with the plugin's Docker files, and permissions could change without warning.

    Do this: - /srv/dev-disk-by-uuid-9d43cda9-20e5-474f-b38b-6b2b6c03211a/appdata/jellyfin/config:/config

    This way, the permissions will remain the same as the container creates them.

    • Official Post
    Quote from chente

    I believe I should protect persistent data and compose files equally. Both contain sensitive data, passwords, etc.

    If you go down that road, it is possible that all volumes have sensitive data. My point is that everything in the compose file directories was meant to be plugin generated content and that is why owner/permissions was enforced. Anything in the compose file directory should not be container generated since the plugin enforced permissions on the parent directory may not be correct.


    Quote from chente

    The CHANGE_TO_COMPOSE_DATA_PATH variable can be useful in some simple systems, but not all. The fact that there is only one variable is somewhat limiting. In basic systems with a single file system, it will work without problems, but if there are multiple file systems with data, the user would have to group them to use that variable in all containers, for example with mergerfs, and that would complicate things. Alternatively, it could be combined with other methods to complete the configurations. That's why I left it optional in the document, focusing instead on the use of global variables to define paths (or symbolic links, alternatively). The user is given the option to use this variable anyway, and its use and what it does are explained, of course.

    It wasn't designed to be something that could be used by all systems. Since having many shared folder selections on the Settings tab doesn't scale well, I just created one. I don't think using global variables is bad. It was added so people could do whatever they want.


    Quote from chente

    Grouping persistent data folders in the appdata folder seemed like the best way to keep things simple when I started using this plugin. I actually copied the idea from the system Zoki used, which he posted a long time ago on the forum. This system was a perfect fit for the plugin, so I did it that way, and it's worked for me for years without any issues. Therefore, I also wrote the wiki document this way.

    I'm not saying your system is wrong. I don't think I have ever looked at Zoki's guide. I was only saying to not put container/user generated files in the compose file directory. I was suggesting that it would be easier to use the CHANGE_TO_COMPOSE_DATA_PATH since I use it for everything on my system and the example compose files I create. I don't care where anyone puts any files but the plugin will continue to maintain the owner/permissions of the compose file directory since it is creating the directory. So, if the container can't handle the permission change, don't put container or user generated files in it.

    omv 8.0.10-2 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.6 | compose 8.1.3 | cterm 8.0 | borgbackup 8.1.5 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post
    Quote from ryecoaaron

    If you go down that road, it is possible that all volumes have sensitive data. My point is that everything in the compose file directories was meant to be plugin generated content and that is why owner/permissions was enforced. Anything in the compose file directory should not be container generated since the plugin enforced permissions on the parent directory may not be correct.

    It works if you use subfolders as designed in the wiki. You're right that it will fail if you place files directly into that folder. That's what happened to the OP.

    Quote from ryecoaaron

    It wasn't designed to be something that could be used by all systems. Since having many shared folder selections on the Settings tab doesn't scale well, I just created one. I don't think using global variables is bad. It was added so people could do whatever they want.

    We agree on that. I just tried to present a universal method that works on any system.

    Quote from ryecoaaron

    I'm not saying your system is wrong. I don't think I have ever looked at Zoki's guide. I was only saying to not put container/user generated files in the compose file directory. I was suggesting that it would be easier to use the CHANGE_TO_COMPOSE_DATA_PATH since I use it for everything on my system and the example compose files I create. I don't care where anyone puts any files but the plugin will continue to maintain the owner/permissions of the compose file directory since it is creating the directory. So, if the container can't handle the permission change, don't put container or user generated files in it.

    Using the CHANGE_TO_COMPOSE_PATH variable for everything implies, for example, that the persistent data will live on a conventional hard drive. This may not be convenient if you need access to that persistent data to be as fast as possible, as might be the case with a Jellyfin or Plex database. In this case, it's better to use an NVME drive or at least an SSD. The data will reside on a slower, larger-capacity conventional hard drive.

    • Official Post
    Quote from chente

    It works if you use subfolders as designed in the wiki. You're right that it will fail if you place files directly into that folder. That's what happened to the OP.

    Depends on permissions. If you pick "administrator - read/write, users - no access, other - no access", then non-root users won't even be able to access anything even if the subfolder is 777.


    Code
    aaron@omv7dev:~$ sudo mkdir /srv/test
aaron@omv7dev:~$ sudo mkdir /srv/test/test2
aaron@omv7dev:~$ sudo chown root:root /srv/test
aaron@omv7dev:~$ sudo chmod 700 /srv/test
aaron@omv7dev:~$ sudo chmod 777 /srv/test/test2
aaron@omv7dev:~$ ls /srv/test/
ls: cannot open directory '/srv/test/': Permission denied
aaron@omv7dev:~$ ls /srv/test/test2
ls: cannot access '/srv/test/test2': Permission denied


    Quote from chente

    I just tried to present a universal method that works on any system.

    I'm sorry if you thought I was insulting your system. I was not. I just don't like files in the compose file directory. Just my opinion. It doesn't mean anything has to change.


    Quote from chente

    Using the CHANGE_TO_COMPOSE_PATH variable for everything implies, for example, that the persistent data will live on a conventional hard drive. This may not be convenient if you need access to that persistent data to be as fast as possible, as might be the case with a Jellyfin or Plex database. In this case, it's better to use an NVME drive or at least an SSD. The data will reside on a slower, larger-capacity conventional hard drive.

    I'm not sure how selecting a shared folder implies it is on a conventional hard drive. You can pick any storage for a shared folder. As usual, I am in the minority and none of my containers use a ton of space. Every container I run is 100% on nvme.


    I just need to stay away from these docker threads. People should follow the guide. Me posting just confuses people.

    omv 8.0.10-2 synchrony | 6.17 proxmox kernel

    plugins :: omvextrasorg 8.0.2 | kvm 8.0.6 | compose 8.1.3 | cterm 8.0 | borgbackup 8.1.5 | cputemp 8.0 | mergerfs 8.0 | scripts 8.0.1 | writecache 8.1


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Official Post
    Quote from ryecoaaron

    Depends on permissions. If you pick "administrator - read/write, users - no access, other - no access", then non-root users won't even be able to access anything even if the subfolder is 777.

    Hmm... I guess you're right, I've never tried doing that. Maybe I should warn about this on the wiki. Too many warnings and notes :)

    Quote from ryecoaaron

    I'm sorry if you thought I was insulting your system. I was not. I just don't like files in the compose file directory. Just my opinion. It doesn't mean anything has to change.

    Not at all. If I gave you the impression I was offended, I wasn't. I just wanted to clarify this since you clearly stated that you don't agree with what's written on the wiki. The wiki is supposed to be there to help you when users ask questions, so I'd like you to be comfortable with what's there. Every time I make a change there, I try to ask you if you're okay with it; I think you've noticed.

    Anyway, if something needs to be changed, we can discuss it. Depending on the change, it may be more or less laborious to adapt the wiki to what you'd like.

    Quote from ryecoaaron

    I'm not sure how selecting a shared folder implies it is on a conventional hard drive. You can pick any storage for a shared folder. As usual, I am in the minority and none of my containers use a ton of space. Every container I run is 100% on nvme.

    Well, I'm just thinking about the most common systems where there are several conventional hard drives and a fast drive (NVME or SSD) for Docker and KVM. As we've already mentioned, there are many possibilities. A system with a single 4TB NVME drive wouldn't raise these concerns.

    Quote from ryecoaaron

    I just need to stay away from these docker threads. People should follow the guide. Me posting just confuses people.

    If you prefer, we can continue discussing this privately, as we've done so many times. I responded to your post here because it was a direct reference to me, as I wrote that document. No problem.

    I moved all the config files into config folders, and it fixed my database permission errors. Thanks for everyone's help <3

    Quote from chente

    The documentation explains how to do this correctly in several places. I've added a warning about this possible permissions change if done incorrectly. I hope this helps others.

    https://wiki.omv-extras.org/do…cker_in_omv#configuration

    Your guide worked and helped me get my Jellyfin server up and running without issue since it had me put the config files in /config. It's when I tried to setup other containers that I fell on my face, and that had nothing to do with your guide. I had three knowledge gaps that lead to my mistake:

    1. Having no Linux experience, leading to a lack of understanding what's common nomenclature and what's not.
    2. Not understanding how Docker is structured.
    3. Not understanding permissions.

    So when I read the documentation for other containers off linuxserver.io I completely misunderstood what they were saying:

    Code
    /home/user/appdata/heimdall:/config

    I saw the word appdata, and just assumed it was common nomenclature of Linux and so they were referring to the 'appdata' path that I had setup for Jellyfin. If I understood Docker better I might have understood they didn't mean the compose folder, but instead any path of my choosing where I wanted to store the config files. And finally if I understood permissions better it might have set off some red flags that putting the config files directly in the compose folder would essentially lock them since I had selected the owner and group of compose directories and files to be root in the Compose plugin.

    OMV: 7.7.4-1 (Sandworm) | Kernel: Proxmox 6.14.0-2-pve

    Plugins: openmediavault-backup 7.1.4 | compose 7.4.4 | fail2ban 7.0.1 | flashmemory 7.0.1 | kernel 7.1.8 | nut 7.0.7-1 | omvextrasorg 7.0.2 | scripts 7.1 | sharerootfs 7.0-1 | zfs 7.1.3

    • Official Post
    Quote from wkruspe

    Not understanding permissions.

    Understanding how permissions work is the first step on the learning curve. This can help you: https://wiki.omv-extras.org/do…omv7:nas_permissions_omv7

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login