Change source address based on destination IP

We have several "router" machines that gather a lot of external IP addresses on the same host and redirect, NAT or proxy the traffic to the internal network. They also act as routers for the machines on the internal network. This works fine, however I am unable to make the routing table, so I can change the source address, based on the destination a machine from the internal network want to access.

Let's say I have a router, that has public addresses P1 (5.5.5.1/24) and P2 (5.5.5.2/24). All traffic goes through P1, but if necessary, the host is reachable on P2 too. This looks like this and works fine:

> ip addr
...
1: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether aa:bb:cc:dd:ee:11 brd ff:ff:ff:ff:ff:ff
    inet 5.5.5.1/24 brd 5.5.5.255 scope global eth1
    inet 5.5.5.2/24 brd 5.5.5.255 scope global secondary eth1:p2
...

Now I want to use P2 as the source address, if I want to access the Google DNS service for example (8.8.8.8). So I add a row in the routing table like:

> ip route add 8.8.8.8 via 5.5.5.254 dev eth1 src 5.5.5.2
> ip route
...
default via 5.5.5.254 dev eth1
5.5.5.0/24 dev eth1  proto kernel  scope link  src 5.5.5.1
8.8.8.8 via 5.5.5.254 dev eth1  src 5.5.5.2 
...

But this does not work. If I ping 8.8.8.8, the host still uses P1 as the source address, and does not use P2 at all for outgoing connections.

Am I doing it right? I guess not...

The fact that you can ping the destination with the right IP address from the host but not from the internal network (if I read your comment right) suggests that you have NAT enabled for the internal network.

In that case, you have to look at iptables rules and edit the according SNAT or MASQUERADE rules. For example:

iptables -A POSTROUTING -s <your-internal-network> -d 8.8.8.8 -o eth1 \
-j SNAT --to-source 5.5.5.2

Please note, that this will only make the forwarded internal network traffic come from the desired IP (5.5.5.2 in the example), you still need iproute rules/tables to make the same rules apply to the router. (Connections coming from the router itself.)