What is an open DNS resolver, and how can I protect my server from being misused by hackers?

I don't have the strongest background in computer security, but yesterday one of my company servers was shut down by our host.

It's a server assigned a public IP where I host several web-service applications including websites and APIs. I was told that my server "is running an open dns resolver which is being used to relay the denial of service attack to an external entity."

What does this mean? How does this attack work? And how can I protect my system from being abused like this?

In my specific case, the server in question is on Windows Server 2012, and it is serving DNS for an Active Directory domain.

Solution 1:

An "open DNS resolver" is a DNS server that's willing to resolve recursive DNS lookups for anyone on the internet. It's much like an open SMTP relay, in that the simple lack of authentication allows malicious 3rd parties to propagate their payloads using your unsecured equipment. With open SMTP relays, the problem is that they forward spam. With open DNS resolvers, the problem is that they allow a denial of service attack known as a DNS Amplification Attack.

The way this attack works is pretty simple - because your server will resolve recursive DNS queries from anyone, an attacker can cause it to participate in a DDoS by sending your server a recursive DNS query that will return a large amount of data, much larger than the original DNS request packet. By spoofing (faking) their IP address, they'll direct this extra traffic to their victim's computers instead of their own, and of course, they'll make as many requests as fast as they can to your server, and any other open DNS resolvers they can find. In this manner, someone with a relatively small pipe can "amplify" a denial of service attack by using all the bandwidth on their pipe to direct a much larger volume of traffic at their victims.

ArsTechnica did a decent article on the recent DNS Amplification DDoS attack against Spamhaus, and is worth a quick read to get the basics (and a good visual of the amplification).

The simplest way to protect your system from being abused like this is to limit the addresses your server will perform recursive lookups for to your local subnets. (The specifics of which depend on which DNS server you're using, of course).

For example, if I were using BIND 9, and wanted to simply prevent DNS recursion from outside addresses, I would use the following code in my config:

options {
    directory "/var/named/master";
    allow-recursion { 127.0.0.1; 10.0.0.0/8; 192.168.0.0/16; 172.16.0.0/12; };

That line of code tells my BIND server to only process recursive DNS requests for the local loopback address (which I guess I could/should set to the local loopback block, the whole /8) and the 3 Private IPv4 address spaces.

For Windows Server 2012, which you say you're using, you have the options below.

1. Separate your DNS server from your IIS server.

• At least in a perfect world, there's no reason you need to be running DNS on the same box as IIS.
  • Put DNS on an internal box that isn't NATed, so the outside world can't get at it, and let IIS reside on the external-facing box that the rest of the world can get at. You can use dual-homing or firewall rules to selectively allow access to your DNS server from your IIS server.

2. Block external DNS requests with a firewall, such as the built in Windows firewall.

• To my surprise, Windows DNS does not allow you to restrict the addresses to which recursive DNS requests are honored, so this actually the recommended method by Microsoft.
• 
  • Select the DNS rules (TCP and UDP), go to the Remote IP address section and add the subnets in use on your LAN, as well as any public-facing IP addresses of servers that need access to Active Directory. As with the BIND example, IPv4 private address spaces are 127.0.0.0/8 10.0.0.0/8 192.168.0.0/16 and 172.16.0.0/12.

3. Disable recursion.

• I'm honestly not sure what effect this will have on your environment, since you haven't really stated how DNS and AD are configured in your environment, and accordingly, it's the last option.
• 
  1. Open DNS Manager.
  2. In the console tree, right-click the applicable DNS server, then click Properties.
  3. Where?
  4. DNS/applicable DNS server
  5. Click the Advanced tab.
  6. In Server options, select the Disable recursion check box, and then click OK.
     • Since we have a multi-forest environment, and use conditional forwarders for that to work, I'm not going to check that box. Might be something for you to consider as well.