docker.io DNS doesn't work, it's trying to use 8.8.8.8
When the Ubuntu Docker package updated to using systemd, it dropped support for the /etc/default/docker
config file, so the initial solution suggested by rocketman10404 will no longer work (disabling dnsmasq
would still work, but it has the downside of preventing Ubuntu from automatically updating the DNS server).
Fixing in the new daemon.json
config file
Find your network's DNS server:
$ nmcli dev show | grep 'IP4.DNS'
IP4.DNS[1]: 10.0.0.2
Open or create, if it doesn't exist, /etc/docker/daemon.json
and add the DNS servers that you want to use:
# /etc/docker/daemon.json
{
"dns": ["10.0.0.2", "8.8.8.8"]
}
Restart the docker daemon:
$ sudo service docker restart
I wrote an in-depth blog post and also filed a bug about this issue if you'd like more details.
(Originally I solved it by opening up /lib/systemd/system/docker.service and add DNS settings to the ExecStart line, but that's bad - we shouldn't edit systemd files directly.)
I don't use docker myself, so I normally wouldn't butt-in here on a docker question, but I just happened to be reading about it and stumbled on some docker documentation that appears to address this exact problem. To sum-up...
The documentation suggests a few workarounds. The first is to specify the DNS server to be used by the docker daemon for the containers by adding the following line to /etc/default/docker
:
docker_OPTS="--dns 8.8.8.8"
where the DNS provided could be a local DNS server, such as 192.168.1.1 (gateway). Then, restart with
sudo restart docker
An alternative solution involves disabling dnsmasq in NetworkManager by commenting out the config in /etc/NetworkManager/NetworkManager.conf
like so:
#dns=dnsmasq
then, restart both
sudo restart network-manager
sudo restart docker
I ran into this in my situation which is specifically
- Running Docker containers on my local development machine
- Which is connected to a VPN
- Some of our container build scripts do things like run
npm install
from a custom repository on the VPN, inside the container.- This works from a CI pipeline but not from our developer machines, because
npm
can't do a successful DNS lookup - We also had problems with containers that need to do lookups to call external REST APIs
- This works from a CI pipeline but not from our developer machines, because
Ubuntu by default uses dnsmasq
started up by NetworkManager to cache DNS requests, and configures /etc/resolv.conf
to point to this instance on 127.0.1.1
- The VPN client we're using is not NetworkManager compatible and forces it's own
/etc/resolv.conf
which overwrites the NetworkManager config - This configures the DNS servers for the VPN
- Docker shadows your
/etc/resolv.conf
to the container by default- Usually on Ubuntu, it passes the Google DNS servers to the container (because it knows about the
dnsmasq
situation. - But it's happy to pass the VPN DNS server config to the container
- There's no route from the container on the
docker0
network bridge, to the DNS servers over the VPNtap0
adapter.
- Usually on Ubuntu, it passes the Google DNS servers to the container (because it knows about the
- Ergo, all DNS lookups in the container fail because it can't reach the only DNS servers it's supplied with
- Additionally, some networks block requests to the Google DNS servers because they want to be able to snoop all your DNS lookups
The solution :
It would seem to be more elegant to use NetworkManager and it's captive dnsmasq
instance in the way it was designed.
-
Tell Docker to use your
dnsmasq
instance for DNS-
Add or edit file
/etc/docker/daemon.json
to tell docker to use thedocker0
bridge adapter for DNS{ "dns": ["172.17.0.1"] }
-
-
Configure the NM
dnsmasq
instance to listen to the Docker bridges as well, because by default it only listens to 127.0.1.1 - create file/etc/NetworkManager/dnsmasq.d/docker-bridge.conf
# Default Docker bridge interface=docker0 # Other Docker bridges interface=br-*
-
I don't like the rude behaviour of that VPN client and I'd rather only use the DNS at the VPN end for VPN lookups (if you have a polite VPN client that uses NetworkManager configured correctly, you won't have to do this)
- Turn off DNS feature in VPN client (it stops overwriting
resolv.conf
on connect and now all DNS goes throughdnsmasq
again) -
Add a config file to tell
dnsmasq
to direct DNS requests for your domain appropriately - add file `/etc/NetworkManager/dnsmasq.d/vpn-dns.confserver=/myprivatedomain.net/10.0.0.1 # or whatever your private DNS server is
-
Optionally, add a search domain for your domain so you can use short names
- I just added our local domain to the search list in my default network connection
- Turn off DNS feature in VPN client (it stops overwriting
-
Restart NetworkManager and Docker
sudo service network-manager restart sudo service docker restart
At this point your Docker containers should be able to do nslookup
without issues when you're on VPN, for domains both inside and outside your VPN.