How can I detect a DDoS attack using pfSense so I can tell my ISP who to block? [duplicate]

Solution 1:

There are several different types of DDoS so any generic information about them may only be correct for one particular type. For instance, the idea that a DDoS always exhausts your bandwidth is incorrect. What you need to do is analyse (some of) the traffic, determine why it's breaking your site, find a way of identifying it and then decide on an action that can block the traffic.

DDoS traffic probably doesn't look like real traffic but what makes it different is not necessarily the quantity. During a recent DDoS we had, our hosting provider decided to block the top ten highest connecting IP addresses. These, of course, all had nothing to do with the DDoS and one of them was actually the Google bot. They would have blocked our office except that they recognised the IP address. The standard advice to use netstat, sort and uniq to find the highest connections is not necessarily good advice.

Our DDoS was a SYN-flood, which means a couple of things for detection:

1. These are not fully opened connections so depending on how you're measuring connections you might not even see these.
2. There is not much identifying information to go on. Pretty much just source IP address and source port.
3. The source address can (and almost certainly will) be spoofed because they're not expecting a response.
4. You may get lucky and find an unusual TCP flag set.

You might only see each IP address once which would make blocking an IP address you had already seen fairly pointless. In practice, we saw 100,000 unique source IP address over 10 minutes and 140,000 packets per second which means (on average) each IP address came back about once per second. Blocking IP addresses would have been effective in our case. The total bandwidth was only 70Mb/s and since the source addresses were spoofed, the whole DDoS could easily have been pushed out by a single server somewhere.

Since each IP address was only sending roughly one packet per second, this was considerably lower than pretty much any other legitimate IP address. Many of our website home pages are up to a MB in size, meaning thousands of packets for what the user sees as a single request.

Mostly the source ports were randomised but for a few hours it changed and the source port was always 1234. This made it very easy to identify the traffic.

In our case, the problem was that the number of packets per second was greater than what the firewall could handle. Blocking these IP addresses at our firewall wouldn't have worked because the firewall was the problem but if the traffic is getting through the firewall and the web boxes are the problem, blocking at the firewall can help. Our hosting provider was able to block the traffic upstream from our firewall when the source port switched to 1234.

There are other types of DDoS and some of these fill your bandwidth. Ours knocked out our firewall anyway, meaning we couldn't access any part of our infrastructure. For your specific question, you will need some kind of out-of-band way to communicate with your firewall and/or boxes so that if your internet pipe gets filled, you can still get in to diagnose the problem. This is generally a good idea anyway, because there are so many other ways your pipe can go down. I tend to see a lot of ADSL modems in other people's racks when I walk around data centres and I wouldn't be surprised if some of these were for out-of-band communication.

Two other DDoS types I have seen are reflected DNS requests and expensive HTTP requests. The reflected DNS requests are particularly troublesome if you run a DNS resolver because you don't want to block these IP adresses in case you want to receive a real answer from one of them. I would change my hosts to use a new DNS resolver (maybe a free public one) and block all other DNS traffic.

The expensive HTTP requests tend to be targeting your CPU/Memory/IOPS. They also come with a lot of identifying information such as HTTP headers and the source addresses can't be spoofed. A tool like mod_security can do some amazing things identifying and blocking these types of requests. It can even drop the tcp connection rather than sending back an HTTP response.

In short:

1. Get out-of-band access.
2. Know your protocols inside out. (TCP, IP, HTTP, whatever you use.)
3. Know your tools inside out. (pfSense, tcpdump, mod_security, etc.)
4. Know your options. (HTTP 403, pfSense block, ISP firewall block, null route, etc.)