How to bypass OpenVPN per application

I have a linux machine that routes its whole internet traffic through an openVPN connection. It was explicitly configured to work that way.

However, the machine has a few applications that should not use the VPN connection (tun+) and go through the open connection (eth0).

As it's not a routing based on the destination ip, my first though was to create a local socks5 proxy where applications should use to avoid the openVPN channel. However, my attempts to do this with ssh -D or other sw) has failed (probably my fault) and I am now wondering if there is a different and better solution for that.

I recently posted another answer that addresses this specific situation, but questions are not duplicates as this question is more specific to OpenVPN. As some people like to downvote for links to answers rather than putting the answer directly in the question, I'm copy/pasting it here.

I have sooo struggled with this so here's a COMPLETE solution. It's tested on Ubuntu 15 to 19.10. You can especially use it with OpenVPN to route certain apps outside the VPN tunnel interface.

The complete "cgroup" solution

How that works?

• Linux kernel will put the app into a control group. Network traffic from apps in this cgroup will be identified by their class ID at the network controller level.
• iptables will mark this traffic and force it to exit with the right IP
• ip route will process marked traffic in a different routing table, with a default route to whatever gateway IP you want.

Automated script

I've made a novpn.sh script to automate dependencies install & run. Tested on Ubuntu 15 to 19.10.

Start your VPN first.

wget https://gist.githubusercontent.com/kriswebdev/a8d291936fe4299fb17d3744497b1170/raw/novpn.sh
# If you don't use eth0, edit the script setting.
sudo chmod +x novpn.sh
./novpn.sh traceroute www.google.com
./novpn.sh --help

Manual HowTo

First, install cgroup support and tools:

sudo apt-get install cgroup-lite cgroup-tools

You need iptables 1.6.0+. Get iptables 1.6.0 release source, extract it, then run this (--disable-nftables flag will avoid errors) from iptables source dir:

iptables --version
sudo apt-get install dh-autoreconf bison flex
./configure --prefix=/usr      \
            --sbindir=/sbin    \
            --disable-nftables \
            --enable-libipq    \
            --with-xtlibdir=/lib/xtables
make
sudo make install
iptables --version

Now, the real config. Define a control group named novpn. Processes in this cgroup will have a classid of 0x00110011 (11:11).

sudo su
mkdir /sys/fs/cgroup/net_cls/novpn
cd /sys/fs/cgroup/net_cls/novpn
echo 0x00110011 > net_cls.classid

Now, we'll suppose the real interface you want to use for the specific app is eth0 with a gateway IP of 10.0.0.1. REPLACE these by what you really want (get the info from ip route), especially in newer Ubuntu versions where interfaces have weird names. Run still as root:

# Add mark 11 on packets of classid 0x00110011
iptables -t mangle -A OUTPUT -m cgroup --cgroup 0x00110011 -j MARK --set-mark 11

# Force the packets to exit through eth0 with NAT
iptables -t nat -A POSTROUTING -m cgroup --cgroup 0x00110011 -o eth0 -j MASQUERADE

# Define a new "novpn" routing table
# DO THIS JUST ONCE !
echo 11 novpn >> /etc/iproute2/rt_tables

# Packets with mark 11 will use novpn
ip rule add fwmark 11 table novpn

# Novpn has a default gateway to the interface you want to use
ip route add default via 10.0.0.1 table novpn

# Unset reverse path filtering for all interfaces, or at least for "eth0" and "all"
for i in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $i; done

Finally, run your app on the specific interface:

exit
sudo cgcreate -t $USER:$USER -a $USER:$USER -g net_cls:novpn
cgexec -g net_cls:novpn traceroute www.google.com
# Close all Firefox windows first
killall firefox; cgexec -g net_cls:novpn firefox

Or if you want to move an already running process to the cgroup, well... you can't! That seems to be due to the NAT (masquerade) function: iptables -nvL -t nat doesn't match when the cgroup is switched, but iptables -nvL -t mangle does match.

# Get PID of the process (we'll then suppose it's 1234)
pidof firefox
# Add to cgroup - THIS DOESN'T WORK! Silently fails to produce the final result.
sudo echo 1234 > /sys/fs/cgroup/net_cls/novpn/tasks
# Remove - but this works...
sudo echo 1234 > /sys/fs/cgroup/net_cls

Credits: No answer were working as expected, but a mix of them did: chripell answer evolware article Per process routing take 2: using cgroups, iptables and policy routing, How do I make a specific process NOT going through a OpenVPN connection?, Kill switch for OpenVPN on the basis of iptables