Why does Linux answer to ARP on incorrect interfaces?
I have the following Linux networking setup: there is an eth10 network interface with the assigned address 10.11.0.1/24. Then there is a tap0 network interface with assigned dummy address 0.0.0.1/32 (I assigned a dummy address to bring the interface up), and traffic from/to that is controlled by a userspace program that originally created the tap0 interface. On the other side of the tap0 interface, there is a userspace program using it via raw sockets that looks for ARP requests and constructs a response.
Now, when the userspace program constructs an ARP request asking for 10.11.0.1, I expect the the other raw socket userspace program to reply to it. However, I get two replies: one from the raw socket program and another coming from Linux kernel.
Apparently, the Linux kernel deduces that 10.11.0.1 is an address belonging to it and thus replies. However, 10.11.0.1 is not an address of the tap0 interface. It is an address of the eth10 interface.
My question is: why does the Linux kernel do that? Is there any way to disable ARP replies on the wrong interface?
My interim solution to this problem is to use other address than 10.11.0.1 for the raw socket / tap0 purposes. But, because this system is supposed to be a system-level test for an application that can be run on any development machine, I cannot guarantee that there are no IP address clashes with other interfaces. Thus, it would be nice to disable ARP replies on the wrong interface.
Another solution to this problem is to use netmap that reserves the entire interface for the userspace application, preventing the kernel from using it for anything while the userspace application is running. But I would like my tests to run without netmap.
Why do you call the ARP reply "wrong"? The system's IP address can certainly be reached via that interface. This is why the ARP reply is sent to begin with. Not doing so might cause some traffic to flow via less optimal paths, or not at all. For example, tap0 might be a VPN connection, and this ARP reply helps to ensure that traffic to the other IP address will flow correctly through the VPN.
If you really want to do this, you can set the sysctls arp_ignore
and arp_announce
to desired values.
arp_announce - INTEGER Define different restriction levels for announcing the local source IP address from IP packets in ARP requests sent on interface: 0 - (default) Use any local address, configured on any interface 1 - Try to avoid local addresses that are not in the target's subnet for this interface. This mode is useful when target hosts reachable via this interface require the source IP address in ARP requests to be part of their logical network configured on the receiving interface. When we generate the request we will check all our subnets that include the target IP and will preserve the source address if it is from such subnet. If there is no such subnet we select source address according to the rules for level 2. 2 - Always use the best local address for this target. In this mode we ignore the source address in the IP packet and try to select local address that we prefer for talks with the target host. Such local address is selected by looking for primary IP addresses on all our subnets on the outgoing interface that include the target IP address. If no suitable local address is found we select the first local address we have on the outgoing interface or on all other interfaces, with the hope we will receive reply for our request and even sometimes no matter the source IP address we announce. The max value from conf/{all,interface}/arp_announce is used. Increasing the restriction level gives more chance for receiving answer from the resolved target while decreasing the level announces more valid sender's information.
And arp_ignore
is described as:
arp_ignore - INTEGER Define different modes for sending replies in response to received ARP requests that resolve local target IP addresses: 0 - (default): reply for any local target IP address, configured on any interface 1 - reply only if the target IP address is local address configured on the incoming interface 2 - reply only if the target IP address is local address configured on the incoming interface and both with the sender's IP address are part from same subnet on this interface 3 - do not reply for local addresses configured with scope host, only resolutions for global and link addresses are replied 4-7 - reserved 8 - do not reply for all local addresses The max value from conf/{all,interface}/arp_ignore is used when ARP request is received on the {interface}
So, you would want to set arp_ignore
to 1 (or possibly 2) and arp_announce
to 2.
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
For testing, it is probably fine to do this. But a real production system will likely behave in the manner you experienced, and your program needs to be able to deal with that.