DNS at systemd's 127.0.0.53 is ignoring some lookups

Solution 1:

I found the fix that worked for me.

my resolv.conf file was pointing to the wrong place. This seems like a bug in Ubuntu as it happened on my laptop (the machine I first noticed this issue on) and on a fresh install of Ubuntu 18.04 Server.

The Default

$ ls -l /etc/resolv.conf

lrwxrwxrwx 1 root root 39 Apr 26 12:07 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

I deleted this and pointed to the correct file. After rebooting, this solved my issue. And I was even able to switch networks on my laptop and the DNS switched correctly. Of course when on external networks I can't resolve any of my local machines but that is expected. As soon as I switch back to my local network, all the local machines resolve correctly because my router is the DNS.

The Fix

$ sudo unlink /etc/resolv.conf
$ sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
$ ls -l /etc/resolv.conf

lrwxrwxrwx 1 root root 32 May 29 08:48 /etc/resolv.conf -> /run/systemd/resolve/resolv.conf

$ sudo reboot

After that, everything worked as I expected and 127.0.0.53 is no longer being used at all.

The Correct Results

$ nslookup web1

Server:     192.168.1.1
Address:    192.168.1.1#53

Name:   web1
Address: 192.168.1.107

$ nslookup google.com

Server:     192.168.1.1
Address:    192.168.1.1#53

Non-authoritative answer:
Name:   google.com
Address: 172.217.7.174
Name:   google.com
Address: 2607:f8b0:4004:80e::200e

Solution 2:

Your resolv.conf file was not pointing to the wrong place — ../run/systemd/resolve/stub-resolv.conf is where it's supposed to point at by default.

The problem is that systemd-resolved does not pass non-dotted names onto DNS. Apparently this is working "as designed". See this github issue which states that "resolved will never allow single-label lookups to leak onto unicast DNS".

Whether or not you agree with the reasoning in that github issue, there is a way to fix this. It doesn't even require making any changes to the default setup on your Ubuntu machine:

1. First, your LAN's DNS must have a domain name.

   If you're using dnsmasq, add the following to /etc/dnsmasq.conf on your DNS server:

   expand-hosts
   domain=your-domain # replace "your-domain" with domain of your choice
   

   You should now be able to resolve LAN hostnames if your add the domain:

   nslookup web1.your-domain
   

2. Second, make sure the name for your LAN's domain is also set in your DHCP server if it's different from your DNS server. On my DHCP server (my router), this setting is just called "Domain Name".

   If you then renew your DHCP lease on your Ubuntu box, you should see a search directive appear in /run/systemd/resolve/stub-resolv.conf:

   nameserver 127.0.0.53
   search your-domain
   

Now looking up web1 will expand it to web1.your-domain, which will then resolve using DNS.

$ nslookup web1
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   web1.your-domain
Address: 192.168.1.107

Note that if you use dig instead of nslookup, dig doesn't use the search path by default — use its +search option to enable that.