Configuring multiple Ethernet connections in a linux server
I have the following topology
.───────────────.
┌──────────────────┐ _.─' `──.
│ │ ,─' '─.
│ ISP Router │ Dynamic IP ; :
│ with NAT ├─────────────────────────────────────────────────────────: Internet ;
│ │ ╲ ╱
└─────────┬────────┘ '─. ,─'
│ `──. _.─'
│ 10.0.0.1 `──────┬──────'
│ │
.──────┴────. .─────┴─────.
,─' '─. ┌─────────────────────────┐ ,─' '─.
,' `. │ │ ,' `.
; LAN : Static DHCP┌──┴───┐ ┌──┴───┐ Static IP ; Public Net :
: ;─────────────┤ eth1 │ My Server │ eth0 ├────────────: ;
╲ 10.0.0.0/24 ╱ 10.0.0.5 └──┬───┘ └──┬───┘ 1.2.3.5 ╲ 1.2.3.4/20 ╱
`. ,' │ Debian Buster │ mydomain.com `. ,'
'─. ,─' │ │ '─. ,─'
`─────────' └─────────────────────────┘ `─────────'
Because of historical reasons, my ISP uses a separate network to provide static IPs.
This means I have to connect my server directly to a special connection that is directly exposed to the internet (in the diagram this is eth0
with IP 1.2.3.5
).
However, I use this same server as a NAS in my home and since that's a separate network altogether I connect it using a different interface eth1
and create a static DHCP allocation with address 10.0.0.5
. This way, other devices in the home network can find it and access the network mounts.
Lastly, I have a DNS record with my domain, in this example mydomain.com
, pointing to 1.2.3.5
.
I'm running into a couple of issues:
- When I try to access
mydomain.com
from my LAN it doesn't work. I believe this is because the Server is getting the packet from interfaceeth0
but then replying througheth1
and the other device is dropping the response since it's from a different IP. - On boot the Server should pick
1.2.3.4
as the network to direct internet traffic to so external access works throughmydomain.com
. However, since botheth0
andeth1
have internet access through the respective gateways, it sometimes boots and picks10.0.0.1
as the default gateway and then accessingmydomain.com
doesn't work. I believe this is a similar issue with traffic entering througheth0
but leaving througheth1
.
While I'm familiar with basic networking stuff, I don't have experience with having a server connected to multiple networks at once like this. Ideally I'd like to configure the server so that:
- All internet traffic exits through
eth0
- All traffic from entering
eth0
is replied througheth0
- All traffic from entering
eth1
is replied througheth1
My server runs Debian, I'd appreciate any help.
EDIT: So after disabling the eth1 gateway and playing a bit with tcpdump
I was able to confirm that traffic is entering from eth0 and leaving eth1. The issue is that the ISP is not NATing traffic going to the endpoint 1.2.3.5
and then the server is identifying the 10.0.0.X
addresses as located on interface eth1
.
This is a routing issue which can be solved with source-based routing or policy routing. This involves two routing tables, one for each source address:
See http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.rpdb.multiple-links.html for more details.