Bridged networking in KVM/Qemu LAN addressed packets dropped
Summary: In a bridged KVM/QEMU configuration, network packets destined to guest VM do not get there.
Configuration: The host is an up to date Ubuntu 20.04.2 LTS server; The guest is any of 3 VMs, a very old 16.04 Ubuntu server, an old Ubuntu 20.04 desktop, and a brand new Ubuntu 21.04 desktop. The first 2 VMs are being converted from non-bridged, NAT'd, and the 3rd was created specifying bridged networking. Ultimately the VMs will get their IP addresses from the main LAN, via DHCP, but for now and for better debugging information, they are using static IP addresses.
The host bridge definition, /etc/netplan/01-netcfg.yaml, (this is one of many many attempts):
# This file describes the network interfaces available on your system
# For more information, see netplan(5).
network:
version: 2
renderer: networkd
ethernets:
enp3s0:
dhcp4: no
bridges:
br0:
# interfaces: [ enp3s0 ]
dhcp4: yes
# dhcp6: no
# link-local: [ ]
interfaces:
- enp3s0
# parameters:
# stp: true
# forward-delay: 4
The virtual stuff, /etc/libvirt/qemu/networks/br0.xml:
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
virsh net-edit br0
or other application using the libvirt API.
-->
<network>
<name>br0</name>
<uuid>40a8752c-d074-4802-bae8-b0aef95d9c99</uuid>
<forward mode='bridge'/>
<bridge name='br0'/>
</network>
Notes: many versions of bridge .xml files have been tried, including different names, and different references use differing techniques. The Ubuntu Serverguide references this which says the name and bridge name have to be the same, but other references do not have them the same. Once the bare file has been created with nano, these commands:
virsh net-define br0.xml
virsh net-autostart br0
virsh net-start br0
were used to add and configure it. The default, NAT'd way was unlinked, from the autostart directory, so as not to. Eventually, it was undefined. The result:
$ virsh net-list --all
Name State Autostart Persistent
----------------------------------------------
br0 active yes yes
At this point, there are no iptables rules at all after re-boot. However the VMs do not have network access. Note that some references mention special iptables rules and special attributes for the br_netfilter module, all of which have been tried. This question is already long enough without going into all the detail of variations tried here.
Debug Details: No matter the configuration variant, the base issue is always the same, VM destined packets do not seem to arrive at the host, at least as seen by tcpdump. However, broadcast type packets do arrive and do get to the client VM.
This example will use 192.168.111.59 (MAC: 52:54:00:60:ea:0e), the 16.04 server VM, and 192.168.111.132, a raspberry-pi, on the LAN. The 20.04 host server is at 192.168.111.136. The netmask is 24 bits, 255.255.255.0. The gateway and DHCP server is a Debian server (on which bridged guest VMs work fine, by the way).
First tpcudmp as seen from the raspberry-pi during ping:
doug@rpi2:~ $ sudo tcpdump -n -tttt -i eth0 ether host 52:54:00:60:ea:0e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
2021-04-23 08:33:19.363553 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 08:33:19.487239 IP 192.168.111.132 > 192.168.111.59: ICMP echo request, id 27848, seq 14, length 64
2021-04-23 08:33:20.363542 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 08:33:20.527250 IP 192.168.111.132 > 192.168.111.59: ICMP echo request, id 27848, seq 15, length 64
2021-04-23 08:33:21.567215 IP 192.168.111.132 > 192.168.111.59: ICMP echo request, id 27848, seq 16, length 64
2021-04-23 08:33:22.607228 IP 192.168.111.132 > 192.168.111.59: ICMP echo request, id 27848, seq 17, length 64
2021-04-23 08:33:23.372351 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 08:33:23.647228 IP 192.168.111.132 > 192.168.111.59: ICMP echo request, id 27848, seq 18, length 64
2021-04-23 08:33:24.371431 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 46
Observe the VM is sending out packets just fine, as seen via all the ARP activity. However it never replies to anything. Now lets watch the same activity from the host, noting tcpdump output is the same for any of interfaces br0 or enp3s0 or vnet0.
$ sudo tcpdump -n -tttt -i br0 ether host 52:54:00:60:ea:0e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
2021-04-23 08:40:38.837608 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:39.837159 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:40.837122 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:43.842985 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:44.840895 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:45.840991 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:48.848508 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:49.848895 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:50.848871 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:51.514011 ARP, Reply 192.168.111.59 is-at 52:54:00:60:ea:0e, length 28
2021-04-23 08:40:52.928400 ARP, Reply 192.168.111.59 is-at 52:54:00:60:ea:0e, length 28
2021-04-23 08:40:53.853881 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-23 08:40:54.852472 ARP, Request who-has 192.168.111.1 tell 192.168.111.59, length 28
Observe every so often the VM does respond, but later on we'll see that it is to a broadcast packet. It also seems as though a problem is that 192.168.111.1 isn't responding. It is, and for whatever reason, that packets are not seen here at the tcpdump level. Also notice no ICMP packets from the raspberry-pi. Now, show the gateway responding (this is "br0" on a different computer. EDIT: replaced with a better capture example, hence the different timestamps):
$ sudo tcpdump -n -tttt -e -i br0 ether host 52:54:00:60:ea:0e
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on br0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
2021-04-23 22:25:17.434415 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 22:25:17.434432 xx:xx:xx:xx:xx:xx > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at xx:xx:xx:xx:xx:xx, length 28
2021-04-23 22:25:20.440843 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 22:25:20.440859 xx:xx:xx:xx:xx:xx > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at xx:xx:xx:xx:xx:xx, length 28
2021-04-23 22:25:21.438316 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 22:25:21.438332 xx:xx:xx:xx:xx:xx > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at xx:xx:xx:xx:xx:xx, length 28
2021-04-23 22:25:22.438266 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 22:25:22.438283 xx:xx:xx:xx:xx:xx > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at xx:xx:xx:xx:xx:xx, length 28
2021-04-23 22:25:25.446312 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 22:25:25.446329 xx:xx:xx:xx:xx:xx > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at xx:xx:xx:xx:xx:xx, length 28
2021-04-23 22:25:26.446195 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-23 22:25:26.446211 xx:xx:xx:xx:xx:xx > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at xx:xx:xx:xx:xx:xx, length 28
Observe the VM outgoing packets, and ARP does complete. I can't figure out how to copy and paste from the VM, which I communicate with via VNC, but it shows some completed, but stale ARP entries in response to the ip neigh command and tcpdump shows some ARP and broadcast packets from the LAN.
Other information (MACs not relevant to this question have been hidden):
$ brctl show br0
bridge name bridge id STP enabled interfaces
br0 8000.3c7c3f0d9983 no enp3s0
vnet0
$ brctl showmacs br0
port no mac addr is local? ageing timer
1 xx:xx:xx:xx:xx:xx no 0.00
1 3c:7c:3f:0d:99:83 yes 0.00
1 3c:7c:3f:0d:99:83 yes 0.00
2 52:54:00:60:ea:0e no 1.68
1 xx:xx:xx:xx:xx:xx no 2.14
1 xx:xx:xx:xx:xx:xx no 36.84
1 xx:xx:xx:xx:xx:xx no 89.57
1 xx:xx:xx:xx:xx:xx no 226.51
1 xx:xx:xx:xx:xx:xx no 13.28
1 xx:xx:xx:xx:xx:xx no 165.68
1 xx:xx:xx:xx:xx:xx no 165.68
1 xx:xx:xx:xx:xx:xx no 265.02
1 xx:xx:xx:xx:xx:xx no 27.62
2 fe:54:00:60:ea:0e yes 0.00
2 fe:54:00:60:ea:0e yes 0.00
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
link/ether 3c:7c:3f:0d:99:83 brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3c:7c:3f:0d:99:83 brd ff:ff:ff:ff:ff:ff
inet 192.168.111.136/24 brd 192.168.111.255 scope global dynamic br0
valid_lft 51547sec preferred_lft 51547sec
7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UNKNOWN group default qlen 1000
link/ether fe:54:00:60:ea:0e brd ff:ff:ff:ff:ff:ff
EDIT: interestingly, all ARP packets from my D-Link AC2600 Wi-Fi Gigabit Router, configured as a switch, always appear on the host and get to the VM and are replied to:
$ sudo tcpdump -n -tttt -e -i br0 ether host aa:aa:aa:aa:aa:aa
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
2021-04-23 22:45:51.463524 aa:aa:aa:aa:aa:aa > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.59 (ff:ff:ff:ff:ff:ff) tell 192.168.111.58, length 46
2021-04-23 22:45:51.463631 52:54:00:60:ea:0e > aa:aa:aa:aa:aa:aa, ethertype ARP (0x0806), length 42: Reply 192.168.111.59 is-at 52:54:00:60:ea:0e, length 28
2021-04-23 22:46:51.466955 aa:aa:aa:aa:aa:aa > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.59 (ff:ff:ff:ff:ff:ff) tell 192.168.111.58, length 46
2021-04-23 22:46:51.467030 52:54:00:60:ea:0e > aa:aa:aa:aa:aa:aa, ethertype ARP (0x0806), length 42: Reply 192.168.111.59 is-at 52:54:00:60:ea:0e, length 28
2021-04-23 22:47:51.466889 aa:aa:aa:aa:aa:aa > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.59 (ff:ff:ff:ff:ff:ff) tell 192.168.111.58, length 46
2021-04-23 22:47:51.466965 52:54:00:60:ea:0e > aa:aa:aa:aa:aa:aa, ethertype ARP (0x0806), length 42: Reply 192.168.111.59 is-at 52:54:00:60:ea:0e, length 28
2021-04-23 22:48:51.479096 aa:aa:aa:aa:aa:aa > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.59 (ff:ff:ff:ff:ff:ff) tell 192.168.111.58, length 46
2021-04-23 22:48:51.479178 52:54:00:60:ea:0e > aa:aa:aa:aa:aa:aa, ethertype ARP (0x0806), length 42: Reply 192.168.111.59 is-at 52:54:00:60:ea:0e, length 28
EDIT 3 - New configuration test: In an attempt to reduce the number of variables the following was done:
- An about to be retired Ubuntu 16.04 server was powered up, providing a new isolated LAN.
- the host Ubuntu 20.04 server was connected directly to the LAN side NIC of the 16.04 server. No switches were involved at all, just one long Ethernet cable.
- Things were tested, and seemed to work fine. Access to everything was via ssh from my main LAN out through my main static WAN IP and back in via my test WAN static IP to the old 16.04 server. Then a chained ssh session from there to the 20.04 host server.
- The Ubuntu 16.04 VM client was started on the host.
- "ping" was attempted from the old 16.04 gateway server to the client.
- The results were the same as the original configuration.
tcpdump on gateway old 16.04 server:
doug@DOUG-64:~$ sudo tcpdump -n -tttt -e -i enp2s0 ether host 52:54:00:60:ea:0e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp2s0, link-type EN10MB (Ethernet), capture size 262144 bytes
2021-04-26 15:10:00.701941 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-26 15:10:00.701965 00:19:b9:0d:af:fa > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at 00:19:b9:0d:af:fa, length 28
2021-04-26 15:10:01.699156 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-26 15:10:01.699169 00:19:b9:0d:af:fa > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at 00:19:b9:0d:af:fa, length 28
2021-04-26 15:10:02.699141 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-26 15:10:02.699154 00:19:b9:0d:af:fa > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at 00:19:b9:0d:af:fa, length 28
2021-04-26 15:10:05.707404 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-26 15:10:05.707417 00:19:b9:0d:af:fa > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at 00:19:b9:0d:af:fa, length 28
2021-04-26 15:10:06.707097 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-26 15:10:06.707110 00:19:b9:0d:af:fa > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at 00:19:b9:0d:af:fa, length 28
2021-04-26 15:10:07.707094 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.111.1 tell 192.168.111.59, length 46
2021-04-26 15:10:07.707107 00:19:b9:0d:af:fa > 52:54:00:60:ea:0e, ethertype ARP (0x0806), length 42: Reply 192.168.111.1 is-at 00:19:b9:0d:af:fa, length 28
tcpdump on 20.04 host server:
doug@s19:~$ sudo tcpdump -n -tttt -e -i br0 ether host 52:54:00:60:ea:0e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
2021-04-26 15:11:35.801771 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:36.800497 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:37.800491 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:40.807062 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:41.804469 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:42.804444 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:45.812553 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:46.812405 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
2021-04-26 15:11:47.812398 52:54:00:60:ea:0e > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.111.1 tell 192.168.111.59, length 28
As a side note: From my chained ssh session to the 20.04 host server, I can chain again and ssh to the VM client just fine.
Conclusion: Something is wrong right at the the Link Layer on the Ubuntu 20.04 server such that the incoming packets are not even "seen" by tcpdump, nor get to the VM guest. Diagram.
EDIT 4: Comparing to information provided by Christian Ehrhardt a potential difference is on my system is the br0 MAC list might be incorrect, with the fist byte replaced. Note: non relevant MAC deleted, 3 VMs running:
doug@s19:~$ brctl showmacs br0
port no mac addr is local? ageing timer
1 3c:7c:3f:0d:99:83 yes 0.00 <<< enp3s0, br0
1 3c:7c:3f:0d:99:83 yes 0.00 <<< enp3s0, br0
4 52:54:00:22:2f:dc no 5.15 <<< VM 3
2 52:54:00:60:ea:0e no 3.29 <<< VM 1
3 52:54:00:60:ea:3e no 12.67 <<< VM 2
4 fe:54:00:22:2f:dc yes 0.00 <<< vnet2
4 fe:54:00:22:2f:dc yes 0.00 <<< vnet2
2 fe:54:00:60:ea:0e yes 0.00 <<< vnet0
2 fe:54:00:60:ea:0e yes 0.00 <<< vnet0
3 fe:54:00:60:ea:3e yes 0.00 <<< vnet1
3 fe:54:00:60:ea:3e yes 0.00 <<< vnet1
For whatever reason Christian does not show a vnet, or whatever it is called on his system, connection to the bridge (I do not know it it is relevant or not):
$ brctl showmacs br0
port no mac addr is local? ageing timer
2 52:54:00:48:40:69 no 2.36 <- Guest
1 52:54:00:95:e4:2a no 0.00 <- outside system
1 52:54:00:9b:9b:0e yes 0.00 <- Host
1 52:54:00:9b:9b:0e yes 0.00 <- Host
EDIT 5: Similar data to EDIT 4, but from Debian Server with 2 VMs running, which is working properly:
doug@s15:~$ sudo brctl showmacs br0
port no mac addr is local? ageing timer
1 52:54:00:22:2f:dc no 17.85
2 52:54:00:27:1b:5e no 18.48 <<< VM 1
3 52:54:00:27:1b:ae no 2.14 <<< VM 2
1 f4:8c:eb:c8:08:a0 no 18.48
2 fe:71:fa:75:16:93 yes 0.00 <<< tap0 (VM1)
2 fe:71:fa:75:16:93 yes 0.00 <<< tap0
3 fe:e1:c5:2a:c7:e3 yes 0.00 <<< tap1 (VM2)
3 fe:e1:c5:2a:c7:e3 yes 0.00 <<< tap1
EDIT 6: information from networkctl, the thing to note is that the Debian server shows "Master: br0" whereas the Ubuntu server doesn't:
Ubuntu:
doug@s19:~$ networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp3s0 ether enslaved configured
3 br0 bridge routable configured
4 vnet0 ether carrier unmanaged
4 links listed.
doug@s19:~$ man networkctl
doug@s19:~$ networkctl^Cnetworkctl
doug@s19:~$ networkctl status vnet0
● 4: vnet0
Link File: /usr/lib/systemd/network/99-default.link
Network File: n/a
Type: ether
State: carrier (unmanaged)
Driver: tun
HW Address: fe:54:00:60:ea:0e
MTU: 1500 (min: 68, max: 65521)
Queue Length (Tx/Rx): 1/1
Auto negotiation: no
Speed: 10Mbps
Duplex: full
Port: tp
Apr 30 07:40:51 s19 systemd-networkd[530]: vnet0: Link UP
Apr 30 07:40:51 s19 systemd-networkd[530]: vnet0: Gained carrier
Debian:
doug@s15:~$ networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 enp3s0 ether enslaved configured
3 enp1s0 ether routable configured
4 br0 bridge routable configured
10 tap0 ether carrier unmanaged
5 links listed.
doug@s15:~$ networkctl status tap0
● 10: tap0
Link File: /usr/lib/systemd/network/99-default.link
Network File: n/a
Type: ether
State: carrier (unmanaged)
Driver: tun
HW Address: fe:8a:6a:ce:18:9c
MTU: 1500 (min: 68, max: 65521)
QDisc: pfifo_fast
Master: br0 <<<<< Different than Ubuntu
Queue Length (Tx/Rx): 1/1
Auto negotiation: no
Speed: 10Mbps
Duplex: full
Port: tp
Question: What is wrong and how can I get bridged VMs working properly?
Solution 1:
We have the same kvm vlan bridge problem on PowerEdge R740 and Broadcomm 10GB NIC. What we found are
Broadcom Adv. Dual 10Gb Ethernet 21.65.33.33 & kernel 5.4 -> bad Broadcom Adv. Dual 10Gb Ethernet 21.65.33.33 & kernel 5.5 -> good Broadcom Adv. Dual 10Gb Ethernet 21.80.16.95 & kernel 5.5 -> bad Broadcom Adv. Dual 10Gb Ethernet 21.80.16.95 & kernel 5.12 -> bad
However, firmware roll back won't work, so be careful.