Docker Stacks

    Rebooted, so the containers are working but portainer is not (i.e cant access the interface/omv says it exited).


    Code
    root@OMV:~# docker ps -a
CONTAINER ID   IMAGE                              COMMAND                  CREATED        STATUS                      PORTS                                                                                            NAMES
9a85594e571a   dlandon/logitechmediaserver        "/sbin/my_init"          47 hours ago   Up 3 minutes                                                                                                                 lms
f5770fc6cd24   plexinc/pms-docker                 "/init"                  47 hours ago   Up 3 minutes (healthy)                                                                                                       plex
13e74f570260   lscr.io/linuxserver/resilio-sync   "/init"                  47 hours ago   Up 3 minutes                0.0.0.0:8888->8888/tcp, :::8888->8888/tcp, 0.0.0.0:55555->55555/tcp, :::55555->55555/tcp         resilio-sync
5897aa008dae   uroni/urbackup-server:latest       "/usr/bin/entrypoint…"   47 hours ago   Up 3 minutes                                                                                                                 urbackup
75ba5e06fcd2   a1c22f3d250f                       "/portainer"             2 months ago   Exited (128) 29 hours ago   0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 9443/tcp, 0.0.0.0:9200->9000/tcp, :::9200->9000/tcp   portainer
root@OMV:~#
    • Official Post
    Quote from tannaroo

    ok that worked - thanks


    my appdata2 folder also contains my docker storage path. so both docker and portainer are in the same folder. so this is likely to be a recurring issue as I guess portainer is conflicting before docker is running?

    Are you using a pi or some USB drive for that appdata folder

    Your portainer exited 29 hours ago:


    75ba5e06fcd2 a1c22f3d250f "/portainer" 2 months ago Exited (128) 29 hours ago 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 9443/tcp, 0.0.0.0:9200->9000/tcp, :::9200->9000/tcp portainer


    There is something weird going on. You said it was running before.


    Please run:



    Code
    cd /.../AppData2/portainer
docker-compose up -d portainer
docker-compose logs portainer

    This should give some insight. (Did you start it with the -d flag before?)

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Quote from KM0201

    Are you using a pi or some USB drive for that appdata folder

    No pi but am using USB drive for Docker as my understanding its better than a spin drive (as there are lots reads/writes to Docker). I know SSD is even better. That will be my next project.


    EDIT: My mistake my old setup was using USB drive but now Docker/Portainer is on an external drive.

    when I initially setup Portainer this way, I did use -d flag.

    The containers are definitely working just not portainer interface unless I use systemctl restart docker in which case portainer starts up.


    Setting up portainer is causing an error because of the existing portainer and I'm reluctant to remove it and start a new portainer as I just got it working yesterday but I attach the logs - is there another way?


    But I can confirm after another reboot, portainer interface is exited but the containers are working.

    • Official Post

    Are you sure they are actually working? Try accessing a container (say plex, or whatever).. through it's webUI, not just by checking the log... If it opens, try opening a movie file or music or something, to make sure it opens. This sounds suspiciously like the old "racing" problem.. but I thought it had been resolved.

    So you have now two containers names portainer which can not run simultaniously.

    Change the docker-compose-file and give it a differnet


    container_name: portainer2


    and make it listen to a differnet ports:


    ports:
    - 8001:8000 # you do not need this port if not using portainer_Agent on some other host
     - 9001:9000

    and you can start both instances without conflicts.


    You should decide on on or the other, as you can not mange stacks created with one portainer from the other (the files are on different volumes)

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Like 1
    Quote from KM0201

    Are you sure they are actually working? Try accessing a container (say plex, or whatever).. through it's webUI, not just by checking the log... If it opens, try opening a movie file or music or something, to make sure it opens. This sounds suspiciously like the old "racing" problem.. but I thought it had been resolved.

    Tried a couple of movies from my TV - plex app and it works. Tried playing songs from squeezer app and it also works.

    But no portainer UI.

    • Official Post

    If that's the case though, the compose file I gave him at the beginning of this would have never worked (although admittedly it does look like he has two containers named portainer).

    Try to find out, why your portainer exited. docker logs portainer should give you the logs of the old portainer container.

    And docker inspect portainer will give you the config of your portiaier. Check its restart policy.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Quote from Zoki

    So you have now two containers names portainer which can not run simultaniously.

    I'm confused here - I only have one portainer installed. The other portainer command did not install.

    Happy to setup up a new portainer per your instructions and report back

    Code
    root@OMV:/srv/dev-disk-by-uuid-fc008cb9-1532-4231-a49e-88721f97ba63/AppData2/portainer# ls
bin    compose             docker_config  portainer.key  tls
certs  docker-compose.yml  portainer.db   portainer.pub

    according to the docker ps we saw earlier, ther was on container named portainer in stopped state (exited 29h aog) and the docker-compose based could not be started because the name was already in use by the stopped one.


    Here the outout again:


    75ba5e06fcd2 a1c22f3d250f "/portainer" 2 months ago Exited (128) 29 hours ago 0.0.0.0:8000->8000/tcp, :::8000->8000/tcp, 9443/tcp, 0.0.0.0:9200->9000/tcp, :::9200->9000/tcp portainer



    So the stopped container is listening on port 9200, which was not in the docker-compose KM0201 posted earlier.


    Please do the docker inspect and post the output so we can see, what kind of beast this thing named portainer is.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Quote from Zoki

    Please do the docker inspect and post the output so we can see, what kind of beast this thing named portainer is.

    Quote from KM0201

    If you log in to portainer (assuming you can again)... can you see two containers named portainer? Even if one is stopped or not running.. there should only be 1.

    I can with : systemctl restart docker


    I've logged in but there is only 1 portainer running. None stopped. No unused images.

    So this /srv/dev-disk-by-uuid-fc008cb9-1532-4231-a49e-88721f97ba63/AppData2/portainerr seems to be the path where the docker-compos.yml is installed you tried to start above.


    The container from inspect was "Created": "2021-11-13T12:34:42.358274692Z",

    but has a named volume mounted:

    Code
            "HostConfig": {
            "Binds": [
                "/var/run/docker.sock:/var/run/docker.sock",
                "portainer_data:/data"
            ],


    and this volume is said to be here:



    You are mixing two different docker installations one on the disk /srv/dev-disk-by-uuid... and the one on /srv/dev-disk-by-label-Chimp/...


    what does docker volume ls say regarding the portainer_data volume?


    Do you still have the docker-compose / stack definitions of your other containers? If yes, you should

    - destroy the portainer docker rm portainer

    - start the docker-compose based portainer by cd /srv/srv/dev-disk-by-uuid.../portainer and docker-compose up -d

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    /srv/dev-disk-by-uuid..... and /srv/dev-disk-by-label-Chimp are the same external drive.


    By uuid - there is Docker, plex, portainer

    Code
    root@OMV:/srv/dev-disk-by-uuid-fc008cb9-1532-4231-a49e-88721f97ba63/AppData2# ls
Docker  Library  plex  portainer

    By label - there just plex (no docker/portainer). Docker used to point here by label on the old installation which crashed over the weekend. So on the new install, I changed it by uuid (per above).


    Code
    root@OMV:/srv/dev-disk-by-label-Chimp/AppData2# ls
plex

    But this would suggest there are somehow 2 volumes - so can I find out which one is which and just delete the wrong one?


    Code
    root@OMV:~# docker volume ls
DRIVER    VOLUME NAME
local     96ddc8e2e81465c5284bbe0283cd0b4031c8a1ee0d4adc4f1723381b2c598373
local     b4735ee123d12add7cf838f58d6daff597fa3351822ea780e94141c1de72ec93

    According what I see, they are not the same.


    The output of

    ls -la /srv 

    and

    lsblk

    please.


    You have two different locations for plex and some other things going wild. OMV changed the way how disks are mounted form by-label to by-uuid some time ago. So docker is mixing up the two locations. It does not recognize the portainer_data volume at the by-label directory which is used by the portainer container.


    This is an explanation of what we have seen.


    As you only have 4 containers, it would be best to recreate them all:

    1. Modify the docker-compose file for portainer in ...-dev-by-uuid.../portainer and give it a differnet container_name and remove port 8000.
      It should start wit hthe default port 9000, as the old one used 9200
      start it
    2. Start the old portainer container with docker start portainer.
    3. Open both portainer UIs in the browser and make sure, bothe are usable
    4. copy the stacks from the old container (port 9200) to the new one (port 9000) on by one one.
      Ofter copying, stop and destroy the container in the old portainer and start it in the new one
    5. make sure the copied containers work
    6. reboot to check if the new portainer (and the others start)
    7. remove the old portainer by docker stop portainer docker rm portainer

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    ok this is interesting. I was aware that OMV had changed mountpoints by label to by uuid. My old installation / stacks were all mapped by label. But in the current fresh installation, I changed all of my stacks/mapping to by uuid. So I should remove all of those mountpoints by label, as they are superseded/duplicated by the uuid. I may be wrong but I feel I need to remove all of the old mountpoints ?

    THese are no longer mount points, but ordinary directories. So some old dockers containers / scripts might have been writing to it, so check the contents before removing.


    Especially the docker containers have the old mount points in the (internal) config which you can check with docker inspect <container_name>

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

    Quote from Zoki

    THese are no longer mount points, but ordinary directories. So some old dockers containers / scripts might have been writing to it, so check the contents before removing.


    Especially the docker containers have the old mount points in the (internal) config which you can check with docker inspect <container_name>

    I carefully checked all docker inspect containers for any old mount points but couldn't find any. I also checked the old srv label directories many were empty but some had duplicate data - so deleted them.


    Rebooted and portainer works.


    As you said, post #73 had a reference ("log path") to the 'old disk by label' whereas now it refers to 'by uuid'. Not 100% sure why previously just the log path was being referenced to the old label whereas everything else was mapped to the disk by uuid. And after deleting /srv/by-label drives, portainer has updated 'log path' to by uuid - which I guess why it works.


    Anyway thanks again to Zoki and KM0201 for patience and assistance.


    I truly hope this is now resolved.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login