Linux: Blocking Huge List of IPs?

if the addresses are grouped in one or few continuous subnets [ eg few /24s ] - i would create simple prerouting rule that would DNAT all traffic on port 80 from them to something listening on port 81 [ eg vhost from nginx ] that would just serve error page:

iptables -t nat -A PREROUTING -p tcp -s 194.88.0.0/16 --dport 80 -j REDIRECT --to 81

if this is 8-10k unique ips scattered 'all around'... it'll be interesting ;-] - i would still go with above scenario but for matching i would use ipset. something like this:

ipset --create ban iphash
ipset --add ban 80.1.2.3
ipset --add ban 90.4.5.6
ipset --add ban 153.17.18.19
# ....
iptables -t nat -A PREROUTING -p tcp -m set --set ban src --dport 80 -j REDIRECT --to 81

Might take a look at: http://www.hipac.org/ It handles large numbers of rules quite well. If there is a pattern to the IPs you are trying to block, you could use the MATCH rule. pQd's prerouting rule would work well with nf-hipac which uses a hash table to store the rules and is considerably quicker than the default iptables lookups.

You could write the ruleset into deny lines in your location block. To make it simple, you could include a file that included your deny blocks and reload the config on an hourly basis. Since it would be within the nginx config file, it shouldn't impact performance too much.

If you have a list of these addresses in a file, you can try the following in Nginx config:

1 Define an IP group using geo directive (inside http block):

http {
...
  geo $block {
    default 0;
    include blacklist; 
  }
...
}

It means that variable $block will be set to 0 by default.

2 Prepare a blacklist file to look like this:

192.168.0.0/24 1;
192.168.2.0/24 1;
10.0.0.0/8 1;

That is, set variable $block to 1 for these addresses.

3 Inside the `location' block, check this variable and report 500 error if it's 1:

 if ($block = 1) {
   return 500;
 }

PS: The efficiency of this solution is arguable, since the connections from banned IPs are accept()ed and passed to the web server. However, if this tiny overhead is not critical for your host, I would rather do it with Nginx, not with iptables.