iptables - Target to route packet to specific interface?

My home server has two main interfaces, eth1 (a standard internet connection) and tun0 (an OpenVPN tunnel). I'd like to use iptables to force all packets generated by a local process owned by UID 1002 to exit through tun0, and all other packets to exit through eth1.

I can easily mark matched packets:

iptables -A OUTPUT -m owner --uid-owner 1002 -j MARK --set-mark 11

Now, I'd like to put some rule in the POSTROUTING chain (probably of the mangle table) to match packets marked with 11 and send them to tun0, followed by a rule that matches all packets and send them to eth1.

I found the ROUTE target, but that seems to only re-write the source interface (unless I'm reading it incorrectly).

Is iptables capable of this? Do I have to mess around with the routing table (via ip route or just the legacy route commands) instead?

Edit: I thought that maybe I should provide more information. I have no other iptables rules at present (although I may create some rules to carry out unrelated tasks in the future). Also, the output of ip route is:

default via 192.168.1.254 dev eth1  metric 203
10.32.0.49 dev tun0  proto kernel  scope link  src 10.32.0.50
85.17.27.71 via 192.168.1.254 dev eth1
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.73  metric 203

I haven't touched the routing table - this is just how it is at present (although it looks fairly dirty). I'm sorry, but I don't have the legacy route command installed on this machine.

And the output of ip addr (of course, eth0 and eth2 can be ignored - they're NICs that aren't being used at present):

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 1c:6f:65:2a:73:3f brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast     state UP qlen 1000
    link/ether 00:1b:21:9d:4e:bb brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.73/24 brd 192.168.1.255 scope global eth1
    inet6 fe80::21b:21ff:fe9d:4ebb/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 00:1b:21:6a:c0:4b brd ff:ff:ff:ff:ff:ff
5: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc     pfifo_fast state UNKNOWN qlen 100
    link/none
    inet 10.32.0.50 peer 10.32.0.49/32 scope global tun0

Edit: I've gotten something sorta working, but it's not forwarding marked packets to tun0. Basically, I added a table (11), and used:

ip route add table 11 via 10.32.0.49 dev tun0
ip rule add priority 10000 fwmark 11 table 11

When I just sudo -u user1000 wget -qO- whatismyip.org, I get my home's external IP address, but if I do sudo -u user1002 wget -qO- whatismyip.org, I also get my home's IP address (but I should be getting the IP at the other end of the OpenVPN tunnel).

Running iptables -vL confirms that the packets are getting matched by the marking rule, but they don't appear to be following the routing rule.

EDIT: I've spent a long time on this, and although it still doesn't work, I think I'm a bit closer.

The iptables rule has to be in the mangle table's OUTPUT chain. I think I also need a MASQUERADE rule in the nat table's POSTROUTING chain, to set the source address. However, the re-routing that occurs after OUTPUT's mangle is not working correctly.

I've spent 5 hours on this now, so I'm taking a break and will probably return to it later tonight or sometime tomorrow.

Solution 1:

I've recently hit a similar issue, albeit a slightly different. I wanted to route only TCP port 25 (SMTP) over an OpenVPN tap0 interface, while routing all other traffic (even for the same host) over the default interface.

To do so, I had to mark packets and set up rules for handling it. First, add a rule that make the kernel route packets marked with 2 through table 3 (explained later):

ip rule add fwmark 2 table 3

You could have added a symbolic name to /etc/iproute2/rt_tables, but I did not bother to do so. The number 2 and 3 are randomly chosen. In fact, these can be the same but for clarity I did not do it in this example (although I do it in my own setup).

Add a route for redirecting traffic over a different interface, assuming the gateway being 10.0.0.1:

ip route add default via 10.0.0.1 table 3

Very important! Flush your routing cache, otherwise you will not get a response back and sit with your hands in your hair for some hours:

ip route flush cache

Now, set a firewall rule for marking designated packets:

iptables -t mangle -A OUTPUT -p tcp --dport 465 -j MARK --set-mark 2

The above rule applies only if the packets come from the local machine. See http://inai.de/images/nf-packet-flow.png. Adjust it to your requirements. For instance, if you only want to route outgoing HTTP traffic over the tap0 interface, change 465 to 80.

To prevent the packets sent over tap0 getting your LAN address as source IP, use the following rule to change it to your interface address (assuming 10.0.0.2 as IP address for interface tap0):

iptables -t nat -A POSTROUTING -o tap0 -j SNAT --to-source 10.0.0.2

Finally, relax the reverse path source validation. Some suggest you to set it to 0, but 2 seems a better choice according to https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt. If you skip this, you will receive packets (this can be confirmed using tcpdump -i tap0 -n), but packets do not get accepted. The command to change the setting so packets get accepted:

sysctl -w net.ipv4.conf.tap0.rp_filter=2

Solution 2:

I solved this. The issue was with the routing rules in table 11. Table 11 was getting hit, but the routing rules make it inoperable. This script is what I now use, and it seems to work well (although it's obviously specific to my setup). Also, I created a new table 21 devoted to the main uplink (eth1).

# Add relevant iptables entries.
iptables -t mangle -A OUTPUT -m owner --uid-owner 1002 -j MARK --set-mark 11
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE

# Flush ALL THE THINGS.
ip route flush table main
ip route flush table 11
ip route flush table 21
ip rule flush

# Restore the basic rules and add our own.
ip rule add lookup default priority 32767
ip rule add lookup main priority 32766
ip rule add fwmark 11 priority 1000 table 11
# This next rule basically sends all other traffic down eth1.
ip rule add priority 2000 table 21

# Restore the main table.  I flushed it because OpenVPN does weird things to it.
ip route add 127.0.0.0/8 via 127.0.0.1 dev lo
ip route add 192.168.1.0/24 dev eth1 src 192.168.1.73
ip route add default via 192.168.1.254

# Set up table 21.  This sends all traffic to eth1.
ip route add 192.168.1.0/24 dev eth1 table 21
ip route add default via 192.168.1.254 dev eth1 table 21

# Set up table 11.  I honestly don't know why 'default' won't work, or
# why the second line here is needed.  But it works this way.
ip route add 10.32.0.49/32 dev tun0 table 11
ip route add 10.32.0.1 via 10.32.0.50 dev tun0 table 11
ip route add 0.0.0.0/1 via 10.32.0.50 dev tun0 table 11

ip route flush cache

## MeanderingCode edit (because I can't comment, yet)

Thanks for this answer! It seems as though this could get messy, as you would have to maintain route info here (possibly duplicating, or breaking other things which may want to set routes.

You may be experiencing "weird things" in your routing table from OpenVPN because the server is configured to "push" routes, enabling all traffic to route through the VPN network interface, rather than the "bare" internet. Or your OpenVPN config or whatever script/application sets it up is setting routes.

In the former case, you can edit your OpenVPN configuration and put in a line containing "route-nopull"
In the latter, check configuration for OpenVPN or any wrapper (network-manager-openvpn, for example on many current linux desktop distros)
In either case, eliminating the routing configuration where it's getting set is cleaner and safer than flushing the table, depending on when you run this script and what else your system is doing.

Cheers!