How do policy based routing tables actually work?
My understanding of Policy Based Routing in Linux is that there are three main components, the routing tables, the routes and the rules. I've followed many guides and in each of them this seems to be correct, so I create a route table, I add a route to it and then I add a rule for qualifying traffic to use this table.
This seems logical to me, there's a list of rules and if traffic matches the rule it looks up the route table and that tells it how to get to its destination, however this doesn't seem to work for me and I've now come across a number of other articles where straight after setting up the routing tables, the subjects then add the same routes to the main route table, I'm not sure why this is required, unless my understanding of how it's meant to work is wrong, or it doesn't work.
Here's a good example from the Linux Advanced Routing & Traffic Control HOWTO it has the following excerpt (in case the link disappears):
One creates two additional routing tables, say T1 and T2. These are added in /etc/iproute2/rt_tables. Then you set up routing in these tables as follows:
ip route add $P1_NET dev $IF1 src $IP1 table T1
ip route add default via $P1 table T1
ip route add $P2_NET dev $IF2 src $IP2 table T2
ip route add default via $P2 table T2
Nothing spectacular, just build a route to the gateway and build a default route via that gateway, as you would do in the case of a single upstream provider, but put the routes in a separate table per provider. Note that the network route suffices, as it tells you how to find any host in that network, which includes the gateway, as specified above.
Next you set up the main routing table. It is a good idea to route things to the direct neighbour through the interface connected to that neighbour. Note the `src' arguments, they make sure the right outgoing IP address is chosen.
ip route add $P1_NET dev $IF1 src $IP1
ip route add $P2_NET dev $IF2 src $IP2
Then, your preference for default route:
ip route add default via $P1
Next, you set up the routing rules. These actually choose what routing table to route with. You want to make sure that you route out a given interface if you already have the corresponding source address:
ip rule add from $IP1 table T1
ip rule add from $IP2 table T2
This set of commands makes sure all answers to traffic coming in on a particular interface get answered from that interface.
OK, so I totally don't get why after adding the routes to the specifically created new route table you then go and add them to the main route table, what was the point in that, can anyone explain?
Actually, I think the confusion comes from the fact that those instructions are mixing two topics :
- Dealing with policy routing
- Setting up your standard routing table in a best-practice way
These three lines
ip route add $P1_NET dev $IF1 src $IP1
ip route add $P2_NET dev $IF2 src $IP2
ip route add default via $P1
are there to handle traffic that doesn't get matched by your policy routes for whatever reason (you add a loopback on the router for example, and use it for management). They are not strictly necessary for the PBR to work, but it's best to have them there all the same.