Advanced dynamic routing with external program

Solution 1:

Netfilter (iptables) has queue module to send frames to a userspace program. Libraries for different languages (c, python, perl, etc...) are available to examine packets. After processing a frame you will return an ACCEPT or DROP verdict, the original or modified frame, and an option to set a mark.

My guess that you can use the mark to handle this packet differently in the rest of the netfilter chain and change a routing mark to choose a specific routing table.

This would be a more elegant solution than very low level device handling but may be a performance issue depending on the choice of your userspace implementation.

I have used this in another project to modify incoming DHCP frames from a broken client but never used the mark.

Solution 2:

Userspace routing can be achieved by pointing a default route at a tun device, and having a userspace program examine each received packet. It's an inefficient and brittle approach, but it has been made to work — there was an AODVv2 implementation that worked that way, due to Henning Rogge.

The other option, of course, is to implement your routing protocol within the kernel — most implementations of AODV/DYMO/AODVv2 work that way.

Before embarking on this kind of task, I would recommend considering carefully whether you need to make routing decisions for each individual packet; if possible, a better approach is to manipulate the routing tables dynamically while leaving the actual forwarding to the kernel. An example of what can be achieved using this approach is given in this draft paper. (Disclaimer: I'm a co-author.)