10,000 entries in iptables on CentOS 4GB Ran, dual core 2.3ghz?
Don't worry, it'll be iron age performance, at worst. (grin)
A large number of netfilter rules, which the kernel has to run through for every single packet, will have a negative impact on system performance and resource consumption (both CPU and memory). The nice thing is that there are a number of tricks which you can employ to minimise the problems.
- "Short-circuit" rules for common cases: 90+% of the packets you receive will be part of an existing established or related connection. Thus, you can avoid having to put all those packets through your lengthy filter list by having a
-m state --state ESTABLISHED,RELATED -j ACCEPT
rule before any of your filter list rules. -
Only filter certain ports: if you're running a webserver, but you're really only worried about SSH brute-force attacks, you can avoid having to process all of the HTTP traffic through the filter list by putting the filter list in a separate chain, and then only sending port 22 traffic through the filter, like this:
iptables -N geofilter iptables -I geofilter -s <ip range> -j DROP [etc etc etc] iptables -I INPUT -p tcp --dport 22 -j geofilter
-
"Tree structure" your filter rules: you may have 10,000 filtering rules, but only one of them will ever match. Thus, you can use a tree structure to reduce the number of rules any one packet has to go through, by putting all of the rules for, say, each /8 into its own chain. Extending on the previous
geofilter
example:iptables -N geofilter_1 iptables -N geofilter_2 [etc etc etc all the way to geofilter_223] iptables -I geofilter -s 1.0.0.0/8 -j geofilter_1 iptables -I geofilter -s 2.0.0.0/8 -j geofilter_2 [etc etc etc] iptables -I geofilter_1 -s 1.2.3.0/24 -j DROP iptables -I geofilter_1 -s 1.5.9.0/18 -j DROP [etc etc etc for all rules in 1/8] iptables -I geofilter_2 -s 2.42.0.0/16 -j DROP [etc etc etc for all rules in 2/8] [continue pattern for all /8s]
This means that any packet that needs to go through the filtering rules will need to, at most, traverse 223 rules (all of the
geofilter
rules) plus however many rules are in any of the per-/8 filter lists. (Why there's 223, not 254, 255, or 256 rules in thegeofilter
chain is left as an exercise for the astute reader). You can make this even more efficient by having multiple levels of tree: split on the /4, then on the /8, then on the /12, or something like that. You can add as many levels to match your cost/benefit tradeoff as you like. You can even do it differently for different chains: start with a single level splitting at the /4, then any chain with more than a few hundred rules, split that at the /8, and any chain that still has more than a few hundred rules, split at the /12. -
Aggregate rules: I would place a reasonable wager that the lists of addresses you're using have not been optimally aggregated. Even if they have been for a single country, once you get the lists of several countries together there may be adjacent blocks that come from different countries that you could put together. As an example, say someone in China had
192.0.2.0/25
, and someone in Hong Kong got192.0.2.128/25
(yes, not realistic blocks, but RFC5737 only gives us a /24 for documentation). You can aggregate that into192.0.2.0/24
and save yourself a rule.Once you start doing this, you can often find that you can reduce the number of rules in your list significantly. (Combined with the next rule, you can reduce your rule list by half or more.) Implementing aggregation is easy; the
netmask
tool will take an arbitrary list of blocks and give you back a minimal list of CIDR blocks:netmask -c 192.0.2.0/25 192.0.2.128/25 192.0.3.0/24 192.0.1.0/25 192.0.1.0/25 192.0.2.0/23
Negative rules: Often, you'll find that a large number of small blocks aggregate into a single, much larger block, except for one little chunk in the middle. In some cases, almost an entire /8 or /10 is allocated to one country, except for a few fiddly little /22s that somehow escaped to another part of the world. In that case, you can put
ACCEPT
rules for those couple of little whitelisted blocks, followed by aDROP
rule for the covering larger block. Working out the optimal blocks will require some degree of programming, but it isn't rocket science.
One thing to note: IP blocks change their geolocation fairly regularly, especially in these "end times" of IPv4. Make sure you don't just fire-and-forget this ruleset. Get updated copies of the geographic lists, and rebuild your filter list based on them. Otherwise, one day you'll find that a block you had filtered has been taken on by your own ISP, and you'll lock yourself out of your own server because an IP address you had previously blocked got assigned to you. (True story)
The main issue with iptables
rules is that they are executed in sequence and with a large rule-set odds are quite a few rules have to parsed before a packet is either granted access or rejected.
Womble's answer already explains quite a few strategies to reduce that processing penalty, by cleverly ordering rules and where I agree the most important one is the use of a statefull firewall configuration where only NEW connections are examined against the complete rule-set and once the initial packet establishing that connection has been examined and approved, all subsequent packets in that same connection are granted access.
Assuming you now have a number of DROP rules blocking vistors:
# Source: http://www.ip2location.com/free/visitor-blocker
iptables -A INPUT -s 1.0.1.0/24 -j DROP
iptables -A INPUT -s 1.0.2.0/23 -j DROP
iptables -A INPUT -s 1.1.0.0/24 -j DROP
iptables -A INPUT -s 1.1.2.0/23 -j DROP
iptables -A INPUT -s 1.1.8.0/21 -j DROP
iptables -A INPUT -s 119.15.136.0/21 -j DROP
iptables -A INPUT -s 119.16.0.0/16 -j DROP
iptables -A INPUT -s 119.18.192.0/20 -j DROP
iptables -A INPUT -s 119.18.208.0/21 -j DROP
Those rules can be reduced to a single iptables rule by using the ipset
utility.
An IP set is list of network addresses and/or ranges maintained by the kernel and matching against that is much faster than sequential matching rules in iptables.
Create an IP set first (the manual recommends the type hash:net
for random sized of netblocks):
ipset create blacklist-china hash:net hashsize 4096
Add the CIDR ranges you wish to block:
ipset add blacklist-china 1.0.1.0/24
ipset add blacklist-china 1.0.2.0/23
ipset add blacklist-china 1.1.0.0/24
ipset add blacklist-china 1.1.2.0/23
ipset add blacklist-china 1.1.8.0/21
ipset add blacklist-china 119.15.136.0/21
ipset add blacklist-china 119.16.0.0/16
ipset add blacklist-china 119.18.192.0/20
ipset add blacklist-china 119.18.208.0/21
Your firewall configuration is now reduced to:
iptables -m set --match-set blacklist-china src -j DROP
Yet another way is to use iptables' xt_geoip
module. Though I have not tested that for performance against ipsets
(if I understand the source correctly xt_geoip
uses d&c binary search, while ipsets
use hashes). Advantage of this is probably easy updating.
For example Ubuntu is shipping this by default in xtables-addons-dkms.