Issues like this are caused by net.ipv4.conf.*.rp_filter.

When rp_filter is enabled, the kernel will reject traffic coming in on an interface where the source address of the traffic matches the subnet of another interface on that server.

Many times problems can arise if the setting is enabled on net.ipv4.conf.all.rp_filter, but not on the specific interfaces like net.ipv4.conf.eth0.rp_filter.
The default setting for rp_filter is 0, however many distros override this setting. Because its being manually set by the distro on boot, setting it under net.ipv4.conf.all.rp_filter no longer works, it must be set on the specific interface (such as net.ipv4.conf.eth0.rp_filter).


I think, that there is a problem with bad segmetation of network, if you need ip r a 192.168.200.6 via 192.168.200.250 line in the configuration... If you declare, that on the eth0.200 is network 192.168.200.0/24, why you want to go to PC with IP address 192.168.200.6 via some other PC? It is not very systematic.

Every network have to be routed only by one router. If you have on the net somewhere router, which is routing traffic between 192.168.100.0/24 and 192.168.200.0/24 network, you can have this config on this machine, but you must disable routing on this PC! Your PC will be on both network, but traffic will come through router, which is what PC on the network are expecting.

When PC want to communicate with other, in the clear case they will see on the network address and netmask, which method it will chose. If it have, for example, IP address 192.168.200.20 and netmask is 255.255.255.0, it is this situation (ipcalc is used):

# ipcalc 192.168.200.20/24
Address:   192.168.200.20       11000000.10101000.11001000. 00010100
Netmask:   255.255.255.0 = 24   11111111.11111111.11111111. 00000000
Wildcard:  0.0.0.255            00000000.00000000.00000000. 11111111
=>
Network:   192.168.200.0/24     11000000.10101000.11001000. 00000000
HostMin:   192.168.200.1        11000000.10101000.11001000. 00000001
HostMax:   192.168.200.254      11000000.10101000.11001000. 11111110
Broadcast: 192.168.200.255      11000000.10101000.11001000. 11111111
Hosts/Net: 254                   Class C, Private Internet

In the normal case this computer can contact every PC in the range 192.168.200.1-192.168.200.254 directly - they will send arp request with destination IP to the broadcast address, destination PC will answer it, and PC will have link address (MAC) and IP address of destination PC and can send it packet directly.

If they want to contact PC from another subnet, they will compute (using IP address and netmask), that it cannot contact it directly and will send packet with IP address of destination PC, but with MAC address of gateway (usually default gateway). Gateway have this net directly connected, or will send packet via another gateway... And answer to our computer will go from it's own gateway, not from another computer on the network.

In the normal case every PC on the network need only 3 specification:

  1. IP address
  2. Netmask
  3. Default gateway

Exception from this rules are gateways (or routers), which know about more that one networks and they have default gateway too...

Sometimes you want to have 'multihomed PC', which have on it's interfaces more than one network, but they want to have disabled routing - they didn't routers (or gateways) at all. If PC are expecting packet from gateway and it will come from another PC, it will drop this packet as fake.

There is a way to say to PC on the network, that packets for address 192.168.100.10 will have to have to send via another PC, but it's:

  • not systematic (see above)
  • it's uncomfortable for administrator, because every PC in the 192.168.200.0/24 network must know about it and opposite - every PC in the network 192.168.100.0/24 network must contact 192.168.200.20 via 192.168.100.10 gateway. Isn't it a mess?

Then I can advise you strongly to don't do it. Keep networking as simple as you can and you will have no problems.