Linux bridge network influenced by route table
I was doing an experiment about linux bridge and my network topology is like:
As you can see, there are two hosts located in a LAN, Host1(10.74.68.58) and Host2(10.74.68.47). On Host1, I created a bridge br0 and assigned an IP for it (192.168.3.101). Then I attached eth0 to the bridge:
[[email protected]:~] # bridge link
2: eth0 state UP : <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 2
I set the default route as br0 and it is ok to ping 10.74.68.47:
[[email protected]:~] # ip r
default dev br0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
192.168.3.0/24 dev br0 proto kernel scope link src 192.168.3.101
But things became unexplainable when I set default route to eth0 :
[[email protected]:~] # ip r
default dev eth0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
192.168.3.0/24 dev br0 proto kernel scope link src 192.168.3.101
When eth0 is the default route interface, I try to ping host2 in two different ways:
1, ping 10.74.68.47
Failed. After I checked tcpdump file (captured on interface br0), I found that br0 only received ARP response. So there is no ARP info on interface eth0, thus it can not get the mac of host2. I think this is the right behaviour, is my understanding right?
2, Then I tried ping -I br0 10.74.68.47
I wanted to use -I option to steer clear of default route, but also failed. After I check the tcpdump file (captured on interface br0), I found there is already a pair of icmp echo request and echo reply packet. This confused me a lot. Now that br0 have received echo reply, why can't I ping to host2 successfully?
[[email protected]:~] # ping -I br0 10.74.68.47
2 packets transmitted, 0 received, 100% packet loss, time 1006ms
Can you guys give me some pointers?
Solution 1:
Bridging doesn't work the way you think it works. :-)
A bridge is only concerned with OSI level 2 (Ethernet frames). On this level, IP addresses etc. don't exist. Conceptually, you can think of a bridge as a collection of ethernet interfaces. Each interface is called a port, and a packet that goes into one port comes out on all other ports. (Actually, in the Linux implementation, there's an optimization that keeps a table of seen MAC addresses, but conceptually, it doesn't matter).
So a bridge can connect ("bridge") several ethernet segments into one big segment.
Then what does it mean to "give a Linux bridge an IP address"? In the Linux implementation, the bridge is not a separate hardware device (like they originally were), it's also accessible from the host itself. That means it acts like a kind of "super-ethernet interface" with many ports, but packets that go into the kernel or out from this kernel to or from any of these ports reach the Linux OS under a single IP address.
So as soon as you make an ethernet interface a slave (port) of a bridge, it ceases to have its own address. The only thing that counts is the IP address of the bridge.
In other words, making a bridge with only a single port makes no sense (you could have used the interface by itself). Trying to route packets to a port of a bridge makes no sense (as far as the kernel is concerned, the bridge is a single device).
If you want to play around with a bridge, you need a structure like this:
10.0.2.1/23 10.0.2.2/23 10.0.3.254/23 10.0.3.1/23 10.0.3.2/23
............ ............ ............... ............ ............
. Host A . . Host B . . Host X . . Host C . . Host D .
. . . . . <-- br0 --> . . . . .
. eth0 . . eth0 . . eth0 eth1 . . eth0 . . eth0 .
.....|...... .....|...... ...|......|.... .....|...... .....|......
| | | | | |
-----+--------------+------------+ +------------+--------------+------
<-------- left Segment ---------> <------- right Segment ----------->
Here the left segment with hosts A and B is bridged by host X to the right segment with hosts C and D, and each host is accessible by a single IP address (which is assigned to interfaces or the bridge as a whole).