How to persist “systemd-resolve” configuration for a particular network interface?

Systemd-networkd store network configurations in (at least) three directories, maybe more. The one that seems most relevant to you is:

/etc/systemd/network/ 

(it overrides the other two, namely /lib/systemd/network and /run/systemd/network).

Create a text file in /etc/systemd/network similar to this:

[Match]
Name=enp0s4

[Network]
DHCP=yes
DNS=192.168.1.1

The file can have any name as long as it ends with .network. Mine is called enp0s4.network. Getting the content right is critical. If you get it wrong, bad things happen, like your networking stops working, at least on that interface, until you restore a working config. Which is not easy – I messed up mine and it took me like an hour to sort it out.

For examples how to set up a bridge, read from here on:

man systemd.network |grep -A 42 "Example 3"

The examples given there plus my snippet of code can set up:

Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.78.38.1

Not sure what the syntax is for DNS domain, it should be possible, the man page is like 30 pages long! If you can, skip it – trying Domain= screwed up my system. So find the right syntax or do not put it in unless you can recover.

NOTE: The correct syntax is Domains= not DOMAIN=!

Here are references I used:

• How to check if the network is running: https://superuser.com/questions/1187633/how-to-debug-systemd-networkd?rq=1#comment1807294_1187633
• How to enable if it is disabled: https://askubuntu.com/posts/681768/revisions

Restart systemd-resolved without reboot:

sudo systemctl restart systemd-resolved

FYI: there is a way to tell systemd to let Network Manager handle an interface, as well as leave it unmanaged and run with flat file configurations like /etc/resolv.conf. But I would make it work as is if I were in your shoes.

This sounds plausible but has does not seem to have any effect:

According to the linked manpage man 8 systemd-resolved.service, you have to use the per link configuration file in /etc/systemd/network:

The DNS servers contacted are determined from the global settings in /etc/systemd/resolved.conf, the per-link static settings in /etc/systemd/network/*.network files, the per-link dynamic settings received over DHCP and any DNS server information made available by other system services.

To make your configuration persistent, you have to create the file /etc/systemd/network/lxdbr0.conf:

[Match]
Name=lxdbr0

[Resolve]
DNS=10.78.38.1
Domains=lxd

This question still doesn't have an accepted answer and as I've also had tremendous problems resolving container names on the host, I thought I should provide my solution. I am also doing this, as this page is the first google result, when looking up persistent changes to sytemd-resolved.service. Lastly I want to mention that this problem seems to be specific to Ubuntu >= 18.04 and it's version of systemd-resolved.service.

So, the manpage systemd-resolved.service has been mentioned already. In case of LXD, misleading conclusions have been drawn out of it however. The LXD interface doesn't need to be defined in /etc/systemd/network/<iface>.{conf|network}. The interface is already working and therefore doesn't need to be defined for systemd's network manager. By default LXD already has a working DNS server listening on it's networks first host address. All we need to accomplish is getting systemd-resolved.service to recognize this server.

The manpage (linked above) mentions this for /etc/systemd/resolved.conf:

The default configuration is defined during compilation, so a configuration file is only needed when it is necessary to deviate from those defaults. By default, the configuration file in /etc/systemd/ contains commented out entries showing the defaults as a guide to the administrator. This file can be edited to create local overrides.

I want to highlight This file can be edited to create local overrides. @quat in fact you also refer to this in your actual question:

In order to make the above persistent, I have reviewed the systemd-resolve manpage http://manpages.ubuntu.com/manpages/bionic/man8/systemd-resolved.service.8.html. It suggests to create a /etc/systemd/resolved.conf.d/lxd.conf file but there are no parameters available that would allow me to make such configuration specific to one particular NIC.

1. Adding lxd's dnsmasq service to systemd-resolved.service global dns settings:
I am not understanding your last sentence however. You don't need to specify the NIC. You merely need to specify the LXD dns server address. Using your network information a working config for /etc/systemd/resolved.conf would look like this (uncomment to change defaults):

[Resolve]
DNS=10.78.38.1
#OBFallbackDNS=
Domains=LXD
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStubListener=yes

After saving the file either restart your server or just systemctl restart systemd-resolved.service For me this worked just fine on a fresh Ubuntu-18.04.02 installation. If it doesn't work for you, I presume you either did a non-default lxd install or you are having problems/conflicts due to your os upgrade.

2. UPDATE (NIC specific setup):
According to Simos' Blog the only way to persist $ sudo systemd-resolve --interface lxdbr0 --set-dns 10.78.38.1 --set-domain lxd is creating a custom systemd service initializing the NIC specific configuration at system start. This way is currently the "accepted" approach on "semi-official" linuxcontainers.org

Example mentioned on the blog:
Shell script enabling nic specific dns via systemd-resolve (lxdhostdns_start.sh)

$ cat /usr/local/bin/lxdhostdns_start.sh 
#!/bin/sh

LXDINTERFACE=lxdbr0
LXDDOMAIN=lxd
LXDDNSIP=`ip addr show lxdbr0 | grep -Po 'inet \K[\d.]+'`

/usr/bin/systemd-resolve --interface ${LXDINTERFACE} \
                         --set-dns ${LXDDNSIP} \
                         --set-domain ${LXDDOMAIN}

Shell script disabling nic specific dns via systemd-resolve (lxdhostdns_stop.sh)

$ cat /usr/local/bin/lxdhostdns_stop.sh 
#!/bin/sh

LXDINTERFACE=lxdbr0

/usr/bin/systemd-resolve --interface ${LXDINTERFACE} --revert

Creating a systemd service to manage the two scripts and haven them initialized at system startup:

$ sudo cat /lib/systemd/system/lxd-host-dns.service 
[Unit]
Description=LXD host DNS service
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/local/bin/lxdhostdns_start.sh
RemainAfterExit=true
ExecStop=/usr/local/bin/lxdhostdns_stop.sh
StandardOutput=journal

[Install]
WantedBy=multi-user.target

Last but not least run systemctl enable lxd-host-dns.service and reboot host to verify the changes being persistent by running systemd-resolve --status. The lxbr0 interface should now be in dns scope (Current Scopes: DNS). Considering your network it should look like this:

Link 3 (lxdbr0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 10.78.38.1
          DNS Domain: lxd

I am well aware of it not being the ideal solution and technically it still doesn't give an answer to your question about persisting this config within systemd-resolve. The problem with persisting it however, lays in the fact that the lxbr0 interface isn't managed by netplan but by systemd on Ubuntu18.04. As netplan doesn't know about the interface, you cannot use the aformentioned *.network interface configurations to add dns and domain settings to lxbr0.

This answer is a long long wall of text now but I hope it at least clarifies the problem a little bit. Both solutions resolve container names on an ubuntu host. The second one is more accurate, but if the only requirement is name resolution on the host, making lxd's dnsmasq service globally available via /etc/systemd/resolved.conf is a viable alternative, especially as it only requires two edits in a config file.

For Ubuntu users (18.04, 18.10, 19.04, 19.10)

I followed hours worth of advice to work around, over, under, through, and behind systemd-resolve, including tapping several merry jigs on top of it as it grunted at me.

What finally worked for me was

1) leave systemd-resolve alone. Don't try to modify it, don't disable it, don't try to uninstall it, don't un-link it... Don't ... Otherwise endless misery will be your constant companion. "But I want to save space..." Trust me, just accept the extra kilobytes, and move forward...

2) Then, (re)install resolvconf sudo apt install resolvconf. It will dance with systemd-resolve like they were meant for each other.

3) after a reboot, make changes if needed like you used to through resolvconf's mechanisms, such as adding content to /etc/resolv.conf.d/resolv.conf/head, or otherwise.

I found that the music that these two systems make together, without my having to participate, was ... relaxing.

Secondarily... perhaps you came from ifupdown, where you were able to declare your DNS servers in /etc/network/interfaces along with you other ip settings, and have them actually work. But now, having been subverted by netplan and systemd-resolve, you're trying to fly through the new magic hoops, wasting your precious limited minutes, hours, searching for the new magic sequence to get network working again...

If that is the case, once again, accept a few extra kilobytes (ridiculous, I know!!); just (re)install ifupdown **sudo apt install ifupdown** and work through your /etc/network/interfaces again.

Both legacy systems fully and seamlessly integrate into the new systems which aren't working for you and many other people. All for the price of a few extra kilobytes (and it's worth it).

It would really be gracious if the Uber-Powers would post educational commentary on the nature of their new subsystems while (as well as just before) the new system is being installed, instead of making the user find out that things don't work, and begin a long search for solutions. (Why is Microsoft coming to mind...?)