Networking with Multiple Nics
I have an Ubuntu 12.04 server with 4 Network adapters in it. I need to use each NIC for a separate function. Here is a description of my setup:
etho = 10.234.0.2 netmask = 255.255.255.252 gw = 10.234.0.1 This is on vlan 234
eth1 = 10.235.0.2 netmask = 255.255.255.252 gw = 10.235.0.1 This is on vlan 235
eth2 = 10.236.0.2 netmask = 255.255.255.252 gw = 10.236.0.1 This is on vlan 236
eth3 = 10.237.0.2 netmask = 255.255.255.252 gw = 10.237.0.1 This is on vlan 237
I need to be able to direct traffic to and from the individual IP addresses for separate web services. ie 10.235.0.2 is a website, 10.236.0.2 is a different site, and 10.237.0.2 is a third site. The 1st IP is for management of the server.
I think the issue is a routing issue, but I am new enough to Linux to not have a full understanding of the ins and outs of the routing capabilities.
Here is what is in my /etc/network/interfaces file:
auto lo
iface lo inet loopback
# WWW Management
auto eth0
iface eth0 inet static
address 10.234.0.2
netmask 255.255.255.252
gateway 10.234.0.1
nameseervers 10.230.1.103, 10.230.70.70
# WWW
auto eth1
iface eth1 inet static
address 10.235.0.2
netmask 255.255.255.252
gateway 10.235.0.1
# WTB
#auto eth2
#iface eth2 inet static
#address 10.236.0.2
#netmask 255.255.255.252
#gateway 10.236.0.1
# Moodle
#auto eth3
#iface eth3 inet static
#address 10.237.0.2
#netmask 255.255.255.252
#gateway 10.237.0.1
I have disabled the last two networks just to ease confusion.
Thanks in advance for all of the help and comments and suggestions.
Solution 1:
After doing a normal configuration of just eth0, I came back around and added config for eth1. With just eth0 up, the route table was:
# ip route show
192.168.0.96/27 dev eth0 proto kernel scope link src 192.168.0.126
default via 192.168.0.97 dev eth0 metric 100
But once I brought up eth1, the order of the default route statements determined which interface was always used. As shown below, it happens to choose the eth1 route to the 192.168.1.65 gateway.
# ip route show
192.168.0.96/27 dev eth0 proto kernel scope link src 192.168.0.126
192.168.1.64/27 dev eth1 proto kernel scope link src 192.168.1.93
default via 192.168.1.65 dev eth1 metric 100
default via 192.168.0.97 dev eth0 metric 100
only one gateway statement
The first problem can be the extra 'via 192.168.1.65' default route. It appears there if the eth1 definition in /etc/network/interfaces has a "gateway 192.168.1.65" statement. So remove any extra gateway statement, and bounce the interface:
# ifdown eth1
# ifup eth1
# ip route show
192.168.0.96/27 dev eth0 proto kernel scope link src 192.168.0.126
192.168.1.64/27 dev eth1 proto kernel scope link src 192.168.1.93
default via 192.168.0.97 dev eth0 metric 100
setup new routing table
Create a new, seperate routing table containing a default route appropriate for all traffic flowing out of eth1. The table number here is not important; 101 is simply not the main routing table. Do this with a 'post-up' command on the eth1 configuration in /etc/network/interfaces. Add only one post-up on eth1; Do not add it to any of the eth1: sub-interfaces.
post-up ip route add default via 192.168.1.65 dev eth1 table 101
Bounce eth1. The main routing table is unchanged, and table 101 will contain the via 192.168.1.65 default route if eth1 is up.
# ifdown eth1
# ip route show
192.168.0.96/27 dev eth0 proto kernel scope link src 192.168.0.126
default via 192.168.0.97 dev eth0 metric 100
# ip route show table 101 (ie, table is empty, no output)
# ifup eth1
# ip route show
192.168.0.96/27 dev eth0 proto kernel scope link src 192.168.0.126
192.168.1.64/27 dev eth1 proto kernel scope link src 192.168.1.93
default via 192.168.0.97 dev eth0 metric 100
# ip route show table 101
default via 192.168.1.65 dev eth1
new routing rule
Add a routing rule to use table 101 to select a default route for packets which should go out eth1.
# ip rule add from 192.168.1.64/27 lookup 101
# ip rule show
0: from all lookup local
32765: from 192.168.1.64/27 lookup 101
32766: from all lookup main
32767: from all lookup default
Add the rule to the /etc/network/interfaces file as well:
post-up ip rule add from 192.168.1.64/27 lookup 101
Now make sure to add cleanup to remove the route and rule when the interface goes down:
post-down ip rule del from 192.168.1.64/27
post-down ip route del default via 192.168.1.65 table 101
[EDIT for ubuntu 16.04+]
Like indicated here and from test I've made from this help, ip route2 have changed his structure of commands.
For making work you will have to adapt just a bit to do in the order how the man ip points.
up ip route add default table 101 dev eth1 via 192.168.1.65
up ip rule add from 192.168.1.64/27 lookup 101
down ip rule del from 192.168.1.64/27
down ip route del default table 101 via 192.168.1.65
Or you will end up after a ifdown - ifup command with a error message @ifdown command(standard message to say that the peripherals is not correctly configured), and @ifup the absence of a route in table 101.