OpenVPN bypass on some ports
Solution 1:
After a little more searching, I've found this thread : https://forum.linode.com/viewtopic.php?p=50114&sid=b440414422596bb7dbc96cf7c9ee511f#p50114
I've now modified my "route-up" OpenVPN script as follow, and it's finally working ! I've removed all the others messy rules (iptable PREROUTING, MASQUERADE, etc).
Here is my final "route-up" script :
ip route flush table 100
ip route flush cache
ip rule add from x.x.x.x table 100
ip route add table 100 to y.y.y.y/y dev ethX
ip route add table 100 default via z.z.z.z
Where x.x.x.x is my server's public IP, y.y.y.y/y is the subnet of my server's public IP address, ethX is my server's public Ethernet interface, and z.z.z.z is the default gateway.
Hope this may help someone else.
Solution 2:
After going through the same ordeal myself I found at least one problem with the route-up script.
iptables -t mangle -A PREROUTING ...
should be:
iptables -t mangle -A OUTPUT ...
Read about why here: http://www.iptables.info/en/structure-of-iptables.html
I didn't have to turn on IP forwarding.