Best way to route traffic based on logged in user via specific redundant route?
I have an Ubuntu 20.04 machine with 2 ethernet interfaces with 2 IP addresses each. It's an AWS EC2 instance and each of the 4 IP addresses has an EIP attached to it via NAT. Both interfaces connect to the same internal subnet. The setup looks like this:
EC2 Machine:
- eni1:
- private-IP1 -> public-IP1
- private-IP2 -> public-IP2
- eni2:
- private-IP3 -> public-IP3
- private-IP4 -> public-IP4
All 4 addresses are reachabale from the outside so that seems to be all fine. However for outgoing traffic currently always private-IP1 (and thus public-IP1) gets used. I want to specify that individual SSH users use specific IP addresses, so they'll come from the corresponding public IP when talking to services on the internet i.e.
user1 -> private-IP1
user2 -> private-IP2
user3 -> private-IP3
user4 -> private-IP4
What's the best way of achieving this result?
Solution 1:
The best way to achieve that would be to clone the EC2 instance into four instances, assign each clone one IP address, and let each user use only the machine having the IP address he or she should use.
Solution 2:
I finally solved this myself after pouring many hours into this. In the end I got it to work by adding an ip rule like so:
sudo ip rule add uidrange 1000-1001 lookup if1
sudo ip rule add uidrange 1002-1004 lookup if2
and then adding iptables postrouting rules like this:
$ sudo iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -o interface1 -m owner --uid-owner 1000 -j SNAT --to-source IP1
-A POSTROUTING -o interface1 -m owner --uid-owner 1001 -j SNAT --to-source IP2
-A POSTROUTING -o interface2 -m owner --uid-owner 1002 -j SNAT --to-source IP3
-A POSTROUTING -o interface2 -m owner --uid-owner 1003 -j SNAT --to-source IP3
with if1 and if2 being my custom route tables containing the corresponding default routes. Works well! Maybe this can be useful for someone in the future :)