Correct way to handle security threats to web server on budget [closed]
During our annual security review I was reminded of an incident earlier this year where we received a threat to our organizations web server. It was over a organization policy and threatened to DDoS our site. Fortunately, nothing bad came of it and it turned out to be an empty threat. However, we still notified the CIO, CSO, and CEO and our hosting provider immediately who applauded our response. Due to the nature of our organization(in education) the preemptive response involved many people including coordination with local law enforcement.
Even though our response was plenty for an empty threat it is making me realize how little attack planning the web app has undergone. Right now the setup is:
- A Linode VPS that is not behind the enterprise firewall (there's a long story behind this that isn't worth explaining)
- a PostgreSQL DB on the same server that only allows local connections
- a Nginx server that we are currently following best practices to secure [1]
- SSH access that we are migrating to certificate authentication
- A backup VPS that has all the latest server settings and just needs the latest version of code pushed and database settings migrated (Right now used as a test server but also envisioned as a georedundancy option)
I guess my question can probably be boiled down to what other steps should I take to lock down my server as well as protect from DDoS? We would love to use Cloudflare Business with their DDoS protection, but we don't always need it and $200 a month is a bit steep for the organization. Do I even need this? Is there a solution that allows temporary DDoS protection? If not, what is the best way to maintain stability during/after an attack? Finally, what logging should be implemented so that we can assist law enforcement in the event an attack occurs?
Solution 1:
what other steps should I take to lock down my server as well as protect from DDoS?
- Firewall off any ports and protocols that are unused. Limit access to only trusted IPs where applicable and possible.
- Apply all security patches and updates in a timely fashion
- Implement a network monitor that can alert on suspicious bursts of activity
$200 a month is a bit steep for the organization. Do I even need this?
No. Not until it adds value and becomes a must have.
Is there a solution that allows temporary DDoS protection?
Yes. They may involve a fair amount of upfront time investment in sorting out the complications of implementing their DDoS service. http://www.blacklotus.net/protect/emergency-ddos-protection is one such on-demand service.
If not, what is the best way to maintain stability during/after an attack?
Just sit there and take it. That's the nature of DDoS. You can try and firewall off IPs, but if the attack is really distributed...well that's just like fighting a wildfire with a squirt gun.
Finally, what logging should be implemented so that we can assist law enforcement in the event an attack occurs?
Maintain logs of incoming source IPs and timestamps and any other relevant forensic data. For example if it's a web server, try and log the user agent, the requested resources. Traffic rates, like packets per seconds, and number of request per second are helpful.
DDoS boils down to mathematical analysis. If someone is trying to extort you they're betting that they can disrupt your business enough to force you to pay protection money to prevent it. Scale is a factor, it's easier to take down smaller operators, but they're able to pay less. If you get an email threat - the best course of action is to ignore it. It takes significant botnet resources to initiate and sustain DDoS attacks - they can't just outright blast everyone like spammers. Chances are they're just doing a massive phishing blast looking for easy targets to intimidate. Due to the nature of the the DDoS beast, victims are fairly helpless unless they can deploy a sophisticated packet filtering prevention scheme or have the budget to contract outside services.
Solution 2:
inetplumber's answer is great.
I would add that another option is to configure your app to scale, so that you could handle a larger attack without impacting your user. You can set up a Virtual Private Cloud (VPC) on Amazon AWS, for example, with your PostgreSQL server accessible only from inside your VPC. You can set up a load balancer front end to distribute the load among multiple servers.
The advantage of doing it that way is that you could quickly scale up to 100's (or more) of servers with no upfront investment, and very quickly (if you were already configured), making you a much more difficult target. You would only need to pay for those servers during hours you were actually under attack. When you were not under attack, you would only need to pay for your one web server and one database server.
The hard part would be getting set up, and it is certainly somewhat more complex than your current configuration. Setting up to rapidly scale up would require even more work. But if you are looking for something you could do to plan (especially if, due to other issues, you think you might be a target in the future), that would be it.
Solution 3:
what other steps should I take to lock down my server as well as protect from DDoS?
if your webapp is not interactive and just displays content: use nginx + proxy-cache with a short-cache-time (1-5 minutes is usually ok). this increases performance a lot and forces an attacker to allocate more resources
setup a resticted firewall that filter everything unneeded IN and OUT by setting INPUT/OUTPUT/FORWARD-Policy to DROP; be sure to log every outgoing connectiona-atempt (except for UPD port 53 :)
if you cannot restrict SSH to a static management-ip, use an alternate high port like 22222, this will prevent a lot of "hello mcfyl - someone in there" - knockers
additionally, use fail2ban/denyhosts to protect ssh from brute force attacks
if you have admin-resources: use OSSEC and a lightweight WAF (there's naxsi for nginx, lightweight and not such a PITA as mod_security), but you'll need someone to setup and maintain such installations
implement backups and use your standby-server as a backup-target
keep your webapp-code up-to-date; if you use an open-source-project, sign up to their security-mailinglist.
try to avoid software that is known to have a lot of vulnerabilities
if you use admin-login and/or managament-webapps: establish basic-auth for those logins + ssl (self-signed cert is OK).
use ssh-tunneling whevener you can instead of opening ports, e.g. for development
If not, what is the best way to maintain stability during/after an attack
- keep calm
- have someone with experience at hand for such a case
- have some emergency-plans ready
the most important thing you did already: thinking about it ( and possible countermeasures ) before it happens.