Laggy server, seems iptables is doing the load

Solution 1:

Add a hardware firewall. MIkrotik has terrific new ones - SERIOUSLY, 2 cores for every network port, running at 1.2ghz.

In the meantime you could try to optimize your iptable rules. THat pretty much is it - if rule processing takes so much time, optimize them to be faster.

Solution 2:

There are a few things you can do to optimise your iptables:

1. Remove rules that don't do anything. Any rule without a target ( -j <something> ) doesn't do anything to the packet, so unless you are using these rules for accounting, they are useless and should be removed.

2. Split rules into chains. A packet has to be matched against every rule up until it finds a target or the end of the chain where it uses the policy. If you create chains for groups of IPs that can be matched with a simple rule, you can drastically reduce the number of rules that have to be compared against for each packet. Something like this:

   :group0 ACCEPT [0:0]
   :group1 ACCEPT [0:0]
   :group2 ACCEPT [0:0]
   :group3 ACCEPT [0:0]
   :group4 ACCEPT [0:0]
   -A INPUT -d 10.0.0.0/24 -j group0
   -A INPUT -d 10.0.0.1/24 -j group1
   -A INPUT -d 10.0.0.2/24 -j group2
   -A INPUT -d 10.0.0.3/24 -j group3
   -A INPUT -d 10.0.0.4/24 -j group4
   -A group0 -d 10.0.0.1 -i ! lo -p tcp -m tcp --sport 80
   ...etc 
   

   And then put all the rules for 10.0.0.0/24 in the group0 chain. Assuming you had one rule for each of the 65,536 IP addresses in 10.0.0.0/16, the worst-case number of comparisons a packet would have to go through in this set up is 512. Much better than the 65,536 comparisons every packet has to go through if they are all in the same chain. It doesn't have to be the destination IP range that you split on, it can be the port number or any other aspect of a packet that IPTables can match.

3. If you are using this for accounting purposes, use a mirroring port on your switch to send a copy of all this traffic to a separate box. Have the primary routing firewall run with the minimum number of rules that achieves your security goals. (There may also be a better way of handling accounting other than iptables but that's not my area.)

4. Put targets in for every rule. Unless you want to match all three of the following rules, you might as well have a -j ACCEPT on every rule. If you do want to match all of them, have the ACCEPT on the last one. That way you don't have to do comparisons for rules that will never match.

   -A acctboth -s 93.xyx.40.250 -i ! lo -p tcp -m tcp --dport 80
   -A acctboth -s 93.xyx.40.250 -i ! lo -p tcp
   -A acctboth -s 93.xyx.40.250 -i ! lo
   

5. Consolidate rules. Since the logic in your firewall is very simple, you could replace the whole thing with about 10 rules - just the ones with -j TARGET.

6. Remove duplicate and redundant rules. These rules don't look very well ordered or organised. I would be surprised if you didn't have duplicate rules in there somewhere. I would also be surprised if, with a few thousand IPs, some of the rules you have are for IPs you no longer have. Find these and remove them.

7. Don't combine the chains for INPUT and OUTPUT. Put the -s <IP> rules in the OUTPUT chain and the -d <IP> rules in the INPUT chain and get rid of the -j acctboth rules and the acctboth chain. Again, this halves the number of rules that have to be checked before a match is found.

8. Order the rules so that the most frequently matched targets are listed first. You can see human-readable counts with iptables -L -nv and raw numbers with iptables -L -nvx. If you have packets that benefit more from low latency (such as packets on port 22) put them earlier in the list than packets that benefit less (such as port 25).