Use OpenVPN only for one application/service

I am using a ubuntu server hosted elsewhere. On that server one of the things running is transmission. I was wondering if I could have the VPN only for that application as once I enable the VPN, I lose access to the server as the IP changes.

I have seen namespaced-openvpn but not sure if that would work for this application

(I am using headless Ubuntu)

Thanks

What you're asking for is split tunneling with OpenVPN for the torrent client Transmission and the complexity of that depends on the firewall rules you prefer. I'd have written the basics but since there were already really good detailed articles on how to setup a split tunneling credential with fallback and make only Transmission to follow that credential, I'm only giving some pointers assuming you already have a working OpenVPN config and proper DNS restriction (no leaks).

1. Install sudo apt install iptables resolvconf apt-utils while you may already have some of the programs. Now add a new user to your system that won't have any superuser ability whatsoever, sudo adduser --disabled-login zzz. zzz is the name of this new user. To avoid service permission headache add zzz to your group and yours to zzz.

sudo usermod -aG zzz <username>
sudo usermod -aG <group_username> zzz 

2. Check ip route list and find the line that looks like 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.101. eth01 is the active network interface and 192.168.1.101 is the inet. Yours could be different. Watch out for these as you follow the upcoming scripts.

3. Flush iptables rules sudo iptables --flush. Append a rule that restricts the zzz user to use regular network interface, only works through the OpenVPN tunnel.

sudo iptables -A OUTPUT ! -o lo -m owner --uid-owner zzz -j DROP

Install sudo apt install iptables-persistent and press YES for any permission it asks for.

4. Create a new script file on openvpn config directory sudo nano /etc/openvpn/iptables.sh and add this script.

#! /bin/bash

export INTERFACE="tun0"
export VPNUSER="zzz" # watch out.
export LOCALIP="192.168.1.101"
export NETIF="eth0"

# Look up the first article on "iptables Script for vpn User" section for the script, I don't know if I'm allowed to paste that. 
# The script simply  uses iptables built-in chains to redirect network traffic to VPN.

Make this script executable sudo chmod 755 /etc/openvpn/iptables.sh.
Create an IP routing script sudo nano /etc/openvpn/routing.sh and type

#! /bin/bash

VPNIF="tun0"
VPNUSER="zzz"

# Look up the first article on "Routing Rules Script for the Marked Packets" section for the script, I don't know if I'm allowed to paste that.
# The script simply puts some routing commands to block the traffic when VPN goes down.

Make this script executable sudo chmod 755 /etc/openvpn/routing.sh

5. Add the new zzz user value to the IP routing table sudo nano /etc/iproute2/rt_tables. Just add 200 zzz at the last line.
Additionally you can add a new kernel parameter config as sudo nano /etc/sysctl.d/zzz.conf and put

net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.eth0.rp_filter = 2

This may not be necessary since it's just hardening the security to kernel level.
Reload sysctl and systemd sudo sysctl --system, sudo systemctl daemon-reload. Now check if openvpn is running okay, systemctl status [email protected] and your openvpn end-IP as user zzz, sudo -u vpn -i -- curl ifconfig.co

If the curl shows the right VPN IP you're good to go for the next steps. You shouldn't be disconnected from the server.

6. Stop the transmission-daemon service, sudo systemctl stop transmission-daemon. Create a new directory and a systemd config at sudo mkdir -p /etc/systemd/system/transmission-daemon.service.d && sudo nano /etc/systemd/system/transmission-daemon.service.d/local.conf. Paste the config

[Unit]
After=sys-devices-virtual-net-tun0.device
Wants=sys-devices-virtual-net-tun0.device

[Service]
User=
User=zzz
Group=
Group=zzz

Type=simple

ExecStart=
ExecStart=/usr/bin/transmission-daemon -f --log-error -g /etc/transmission-daemon

Restart=on-failure
RestartSec=5

This will allow transmission service to run only after openvpn service has run. Reload systemd sudo systemctl daemon-reload.

7. Adding proper user, group and permission to transmission

sudo chown -R zzz:zzz /etc/transmission-daemon/ && sudo chmod -R 775 /etc/transmission-daemon/
sudo chown -R zzz:zzz /var/lib/transmission-daemon/ && sudo chmod -R 775 /var/lib/transmission-daemon/

Do the same for all download folders of transmission like sudo chown -R zzz:zzz ~/Downloads && sudo chmod -R 775 ~/Download.
Add the changed directory to transmission settings sudo nano /etc/transmission-daemon/settings.json,

"download-dir": "/home/Downloads",
"umask": 002,

8. Finally turn on sudo systemctl start transmission-daemon.service. If you're using the Transmission Web GUI follow the nginx method. You can try checking if the torrent is working with the openvpn IP here.