Bridging on Linux using TAP devices (Virtuallbox)

Bridging on Linux using TAP devices

Hello, I have got the following schema:

In order to be able to ping the remote hosts, separatehost1 in my case, I bring up bridge device br0 which combine tap0 + eth2 on server and be to bridge local network (192.168.111.0/24) with remote openvpn client(vpn2) but still no luck, could you please suggest what I am doing wrong

I can ping: vpn1 -> vpn2 vpn2 -> vpn1 vpn1 -> separatehost1 I cannot ping separatehost1 from vpn2 (client) and vice versa (Destination Host Unreachable).

Server.conf

#Server config
proto udp
port 1194
persist-key
persist-tun
keepalive 10 60
tls-auth /etc/openvpn/movpn/ta.key 0
remote-cert-tls client
dh /etc/openvpn/movpn/dh2048.pem
ca /etc/openvpn/movpn/ca.crt
cert /etc/openvpn/movpn/server.crt
key /etc/openvpn/movpn/server.key
user nobody
group nogroup
# use ‘group nogroup’ on Debian/Ubuntu
verb 3
daemon
log-append /var/log/openvpn.log
#client-to-client
dev tap0
server-bridge 192.168.111.101 255.255.255.0 192.168.111.128 192.168.111.200

All schema is created using Vagrantfile + net.ipv4.ip_forward = 1.

The script that brings up br0 inteface:

 #!/bin/bash
 br="br0"
 tap="tap0"
 eth="eth2"
 br_ip="192.168.111.101"
 br_netmask="255.255.255.0"
 br_broadcast="192.168.111.255"
 # Create the tap adapter
 openvpn --mktun --dev $tap
 # Create the bridge and add interfaces
 brctl addbr $br
 brctl addif $br $eth
 brctl addif $br $tap
 # Configure the bridge
 ifconfig $tap 0.0.0.0 promisc up
 ifconfig $eth 0.0.0.0 promisc up
 ifconfig $br $br_ip netmask $br_netmask broadcast $br_broadcast

based on my troubleshooting, the server doesn't forward icmp packets to separatehost1, I don't know why...

root@vpn1:/etc/openvpn/movpn# ip -d link show br0
12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:6c:77:40 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    bridge forward_delay 1500 hello_time 200 max_age 2000 ageing_time 30000 stp_state 0 priority 32768 vlan_filtering 0 vlan_protocol 802.1Q bridge_id 8000.8:0:27:6c:77:40 designated_root 8000.8:0:27:6c:77:40 root_port 0 root_path_cost 0 topology_change 0 topology_change_detected 0 hello_timer    0.00 tcn_timer    0.00 topology_change_timer    0.00 gc_timer  150.00 vlan_default_pvid 1 vlan_stats_enabled 0 group_fwd_mask 0 group_address 01:80:c2:00:00:00 mcast_snooping 1 mcast_router 1 mcast_query_use_ifaddr 0 mcast_querier 0 mcast_hash_elasticity 4 mcast_hash_max 512 mcast_last_member_count 2 mcast_startup_query_count 2 mcast_last_member_interval 100 mcast_membership_interval 26000 mcast_querier_interval 25500 mcast_query_interval 12500 mcast_query_response_interval 1000 mcast_startup_query_interval 3124 mcast_stats_enabled 0 mcast_igmp_version 2 mcast_mld_version 1 nf_call_iptables 0 nf_call_ip6tables 0 nf_call_arptables 0 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 
root@vpn1:/etc/openvpn/movpn# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:b4:26:99 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:db:97:af brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:6c:77:40 brd ff:ff:ff:ff:ff:ff
11: tap0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP mode DEFAULT group default qlen 100
    link/ether d6:df:32:8a:b0:5e brd ff:ff:ff:ff:ff:ff
12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:6c:77:40 brd ff:ff:ff:ff:ff:ff

root@separatehosts1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:b4:26:99 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 76182sec preferred_lft 76182sec
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:a5:4b:55 brd ff:ff:ff:ff:ff:ff
    inet 192.168.111.102/24 brd 192.168.111.255 scope global eth1
       valid_lft forever preferred_lft forever

root@vpn1:/etc/openvpn/movpn# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:b4:26:99 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 75325sec preferred_lft 75325sec
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:db:97:af brd ff:ff:ff:ff:ff:ff
    inet 192.168.33.101/24 brd 192.168.33.255 scope global eth1
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
    link/ether 08:00:27:6c:77:40 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a00:27ff:fe6c:7740/64 scope link 
       valid_lft forever preferred_lft forever
11: tap0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 100
    link/ether d6:df:32:8a:b0:5e brd ff:ff:ff:ff:ff:ff
12: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 08:00:27:6c:77:40 brd ff:ff:ff:ff:ff:ff
    inet 192.168.111.101/24 brd 192.168.111.255 scope global br0
       valid_lft forever preferred_lft forever
root@vpn2:/etc/openvpn/movpn# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:b4:26:99 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 75777sec preferred_lft 75777sec
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:d8:ad:47 brd ff:ff:ff:ff:ff:ff
    inet 192.168.33.102/24 brd 192.168.33.255 scope global eth1
       valid_lft forever preferred_lft forever
9: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
    link/ether 2a:2f:98:b3:34:81 brd ff:ff:ff:ff:ff:ff
    inet 192.168.111.128/24 brd 192.168.111.255 scope global tap0
       valid_lft forever preferred_lft forever

VirtualBox by default uses MAC based filtering straight in the virtual switch code. It knows the MAC address of the VM, and doesn't deliver any other packets (besides multicast and broadcast). The reason is of course security - otherwise a VM can try to redirect traffic etc etc.

After several hours, I rebooted all hosts and traffic started going through the tunnel, it seems to me Virtualbox's virtual switch discovered that I have got created additional interfaces and before the reboot was blocking them after the reboot arp requests and all traffic started working fine :)

Hope this answer will save your time.

separatehost1 -> vpn2

root@separatehosts1:~# ping 192.168.111.128
PING 192.168.111.128 (192.168.111.128) 56(84) bytes of data.
64 bytes from 192.168.111.128: icmp_seq=1 ttl=64 time=2.62 ms
64 bytes from 192.168.111.128: icmp_seq=2 ttl=64 time=2.50 ms
64 bytes from 192.168.111.128: icmp_seq=3 ttl=64 time=2.42 ms
64 bytes from 192.168.111.128: icmp_seq=4 ttl=64 time=1.57 ms
64 bytes from 192.168.111.128: icmp_seq=5 ttl=64 time=2.53 ms
^C
--- 192.168.111.128 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4258ms
rtt min/avg/max/mdev = 1.571/2.333/2.624/0.386 ms

vpn2 -> separatehost1

root@vpn2:~# ping 192.168.111.102
PING 192.168.111.102 (192.168.111.102) 56(84) bytes of data.
64 bytes from 192.168.111.102: icmp_seq=1 ttl=64 time=1.21 ms
64 bytes from 192.168.111.102: icmp_seq=2 ttl=64 time=2.02 ms
64 bytes from 192.168.111.102: icmp_seq=3 ttl=64 time=2.15 ms
64 bytes from 192.168.111.102: icmp_seq=4 ttl=64 time=1.71 ms
64 bytes from 192.168.111.102: icmp_seq=5 ttl=64 time=2.63 ms
^C
--- 192.168.111.102 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4009ms
rtt min/avg/max/mdev = 1.214/1.946/2.634/0.474 ms