Docker in Docker DNS Problem

masterElmar · Jul 7th 2023

Dear members of the OMV forum,

after the last two docker compose updates in omv, I am getting error messages regarding DNS when running a docker in docker.

Apparently it changed something when updating, which I would like to learn about.

My docker compose file is meant for a gitlab runner, where the register and actual runner can connect only dind then fails a dns resolution, which still worked before the OMV update?

Code

version: "3.5"


services:
dind:
image: docker:24.0.2-dind
restart: always
privileged: true
environment:
DOCKER_TLS_CERTDIR: ""
command:
- --storage-driver=overlay2
volumes:
- ./data/dind/docker:/var/lib/docker


runner:
restart: always
image: registry.gitlab.com/gitlab-org/gitlab-runner:alpine
depends_on:
- dind
environment:
- DOCKER_HOST=tcp://dind:2375
volumes:
- ./config:/etc/gitlab-runner:z
- ./data/runner/cache:/cache
- /var/run/docker.sock:/var/run/docker.sock


register-runner:
restart: 'no'
image: registry.gitlab.com/gitlab-org/gitlab-runner:alpine
depends_on:
- dind
environment:
- CI_SERVER_URL=${CI_SERVER_URL}
- REGISTRATION_TOKEN=${REGISTRATION_TOKEN}
command:
- register
- --non-interactive
- --locked=false
- --name=${RUNNER_NAME}
- --executor=docker
- --docker-image=docker:24.0.2-dind
- --docker-volumes=/var/run/docker.sock:/var/run/docker.sock
volumes:
- ./config:/etc/gitlab-runner:z

Display More

level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]"

Quote

Will be retried in 3s ...

Using Docker executor with image node:20.2.0-alpine3.18 ...

Using helper image: registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-79704081

Pulling docker image registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-79704081 ...

Using docker image sha256:f30fc6cc2c3a3c5f3ae052ef95ae06faaf8878f0f9f58ddad81986525d03775e for registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-79704081 with digest registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper@sha256:bb764179fa955138995229ea5deeaa77d171aa4bcc10e0f0b0233b1f2e1eb1c2 ...

Pulling docker image node:20.2.0-alpine3.18 ...

Using docker image sha256:04ad56d14de26e3cd429d98d4990eb082c4f30bb0ff5b574bd56b0ca299ea3f6 for node:20.2.0-alpine3.18 with digest node@sha256:59ac6536ba03469adc3847f23a4f223b0418fba21c90168d703f61bd84125989 ...

<u> </u>Preparing environment 00:01Running on runner-fvxqtyaz-project-763-concurrent-0 via 4d4a17032275...

Getting source from Git repository 00:01Fetching changes with git depth set to 20...

Initialized empty Git repository in /builds/*****.*****/testprojekt/.git/

Created fresh repository.

fatal: unable to access '<a href="https://gitlab.dit.htwk-leipzig.de/elmar.kresse/flowmetrics.git/'">https://***********.git/'</a>: Could not resolve host: *********.de

Display More

I would appreciate some links or information on what I can look for or read.

Many thanks in advance

With kind regards

Elmar

FrederikSchack · Jul 7th 2023

I'm a newbie and I have no idea if this is a solution, but if it's about local DNS or mDNS, then I discovered that it works best if all the containers are running on the "host" network. If they are on Bridge, they can't discover each other.

You can add this to the stack to make sure it runs on "host" network:
network_mode: host

raulfg3 · Jul 7th 2023

Quote from FrederikSchack

I'm a newbie and I have no idea if this is a solution, but if it's about local DNS or mDNS, then I discovered that it works best if all the containers are running on the "host" network. If they are on Bridge, they can't discover each other.

You can add this to the stack to make sure it runs on "host" network:
network_mode: host

Host network is not the best aproach. you only need to create a new Network docker.

I name it "my-net" in my docker-compose.yaml files and use it on all bridge dockers, whit that aproach you can resolve by name all your dockers.

PD: is widely documented on docker documentation: https://docs.docker.com/engine…mmandline/network_create/

you can create in OMV WebGUI, on Portainer WebGUi or by CLI.

FrederikSchack · Jul 7th 2023

Quote from raulfg3

Host network is not the best aproach. you only need to create a new Network docker.
I name it "my-net" in my docker-compose.yaml files and use it on all bridge dockers, whit that aproach you can resolve by name all your dockers.

PD: is widely documented on docker documentation: https://docs.docker.com/engine…mmandline/network_create/

you can create in OMV WebGUI, on Portainer WebGUi or by CLI.

Display More

Ok, thanks for the advice

chente · Jul 7th 2023

Quote from masterElmar

after the last two docker compose updates in omv, I am getting error messages regarding DNS when running a docker in docker.

Apparently it changed something when updating, which I would like to learn about.

I'm sure that this change in your DNS behavior is not due to any compose update, the plugin doesn't change anything in that sense. Keep in mind that other updates are done at the same time, and lately (I don't know how long it's been since you last updated) there were some changes in the OMV network settings.

Check the logs for information about those changes. https://www.openmediavault.org/?page_id=1271

If the latest updates cause your container's network settings to clash with OMV's network settings maybe this will help. https://wiki.omv-extras.org/do…_the_same_lan_as_the_host

masterElmar · Jul 8th 2023

Quote from chente

I'm sure that this change in your DNS behavior is not due to any compose update, the plugin doesn't change anything in that sense. Keep in mind that other updates are done at the same time, and lately (I don't know how long it's been since you last updated) there were some changes in the OMV network settings.
Check the logs for information about those changes. https://www.openmediavault.org/?page_id=1271

If the latest updates cause your container's network settings to clash with OMV's network settings maybe this will help. https://wiki.omv-extras.org/do…_the_same_lan_as_the_host

Thanks that exactly what iam searching for. But also thanks for all the others, i learned some new things.
In the future i should read more changelogs between version updates.

chente · Jul 9th 2023

You are welcome. I'm glad it helped you.

Docker in Docker DNS Problem

ryecoaaron Jul 7th 2023

Participate now!

Similar Threads

Making data externally available OpenVPN?

Swag instance

Docker fails to install on clean OMV installation

Docker "installed and not running" a couple of days after OMV update

swag help

Tags