How can I detect and survive being "Slashdotted"?

Solution 1:

1. install munin to monitor load/memory consumption etc and notify on overloads.
2. install monit to restart apache2 if it crashes
3. install nginx as apache2 frontend, it will massively decrease memory requirements under heavy load

Solution 2:

It's worth mentioning that clever caching and low bandwidth modes will be useless if you simply don't have enough bandwidth on your connection, so make sure the connection to your server is fat enough. Don't host it on your home DSL connection, for example.

I speak from experience of being slashdotted. It's not fun when you can't access the Internet at all because thousands of people are simultaneously trying to download photos of a computer your housemate mounted inside a George Foreman grill. No amount of firewalling will save you.

Solution 3:

The basics:

1. Don't try to host high-volume sites on Windows unless you are a true Windows guru. It can be done, but it's a time versus cost issue.
2. Use static content (i.e., no database queries) everywhere you can.
3. Learn about cache-control headers and use them properly for images and other static assets.
4. At the very least, use Apache, but if you can, use lighttpd or another high-performance webserver.

The real answers:

1. Really know your SQL, and spend time analyzing slow queries. Most page loads shouldn't require more than a second of straight queries.
2. Determine where your load really is. If it's a media-heavy site, consider hosting content elsewhere (like Akamai, or some other service). If it's a database-heavy site, consider replication.
3. Know what kind of replication will work for you. If you have a read-heavy site, standard MySQL master/slave replication should be fine. If you have a lot of writes going on, you'll need some kind of multi-master setup, like MySQL Cluster (or investigate 'cascading' or 'waterfall' replication).
4. If you can, avoid calling PHP - i.e. have a cached static (HTML) copy of the page (which is what most of the Wordpress caching plugins do). Apache is much faster serving static files than even the simplest hello world PHP script.

Solution 4:

Here's a rather lengthy but highly informative article about surviving "flash crowds".

Here's their scenario for the situation their proposed solutions address:

In this paper, we consider the question of scaling through the eyes of a character we call the garage innovator. The garage innovator is creative, technically savvy, and ambitious. She has a great idea for the Next Big Thing on the web and implements it using some spare servers sitting out in the garage. The service is up and running, draws new visitors from time to time, and makes some meager income from advertising and subscriptions. Someday, perhaps, her site will hit the jackpot. Maybe it will reach the front page of Slashdot or Digg; maybe Valleywag or the New York Times will mention it.

Our innovator may get only one shot at widespread publicity. If and when that happens, tens of thousands of people will visit her site. Since her idea is so novel, many will become revenue-generating customers and refer friends. But a flash crowd is notoriously fickle; the outcome won't be nearly as idyllic if the site crashes under its load. Many people won't bother to return if the site doesn't work the first time. Still, it is hard to justify paying tens of thousands of dollars for resources just in case the site experiences a sudden load spike. Flash crowds are both the garage innovator's bane and her goal.

One way out of this conundrum has been enabled by contemporary utility computing.

The article then proposed a number of steps the garage innovator can take, such as using storage delivery networks and implementing highly-scalable databases.

Solution 5:

I rewrite all URLs referred by several popular sites to be redirected through the coralCDN.

An example for Apache:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} !^Googlebot
RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?digg\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?slashdot\.org [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?slashdot\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?fark\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?somethingawful\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?kuro5hin\.org [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?engadget\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?boingboing\.net [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?del\.icio\.us [OR]
RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?delicious\.com
RewriteRule ^(.*)?$ http://example.com.nyud.net/$1 [R,L]
</IfModule>