Block 1.4 million IP addresses on VPS

How can I block a list of about 1.4 million IP addresses? I've already tried to do it with iptables PREROUTING, like:

-A PREROUTING -d IP_HERE/32 -j DROP

But with this many records, my bandwidth goes down like crazy when I do a speedtest.

Without blocked IPs in iptables:

1 Gb/s

With blocked IPs in iptables:

3 Mb/s at peak.

I want to use XDP_DROP like here (last step): https://blog.cloudflare.com/how-to-drop-10-million-packets/

But I don't have an idea how to use this. :/ (I'm really bad at programing)

Are there alternatives to this approach?

You should have a look into ipset.

From the official website:

Ipset may be the proper tool for you [...] to store multiple IP addresses or port numbers and match against the collection by iptables.

[...] (Ipset) may store IP addresses, networks, (TCP/UDP) port numbers, MAC addresses, interface names or combinations of them in a way, which ensures lightning speed when matching an entry against a set.

To use it, you need to create an ipset, add the IPs and create an iptables rule to match with the ipset:

ipset create blacklist hash:ip hashsize 1400000
ipset add blacklist <IP-ADDRESS>
iptables -I INPUT -m set --match-set blacklist src -j DROP

A real life example of usage can be found here. Notice that it uses ipset restore instead of going through each IP in a loop because it’s much more faster.

If your list of IPs has overlaps, you may want to preprocess it to convert to IP ranges where possible. Here is an example of a tool to do it. It won't get you better performances with ipset but it will reduce the size of your list.

On a side note, in term of performances, it is very fast and scale without penalty. As the Cloudflare's blog mention, there are faster low level approaches; but it's much more complex and only adds a few bytes per seconds, which, unless you have the scale and ambition of a cloud provider, are not worth the effort.

Frame challenge - what's the shorter list, authorised or blocked addresses?

Rather than denying 1.4 million, simply allow the perhaps ~dozen IPs you want to permit, and default-deny everything.

If the IP addresses operate in a well-defined range, then you can use ufw like this to block traffic:

sudo ufw deny from 192.0.0.0/8 to any

The example above blocks all traffic from 192.0.0.1 to 192.255.255.254, which works out to 16,777,214 addresses and this has zero (noticeable) effect on network throughput.

So long as your IP list is in a workable fashion to generate IP ranges, this may work for you.