How do I improve the performance of my Drupal site with a lot of concurrent users?

I have a Drupal site that may receive a very high hit rate on launch. We've rented a very beefy server for the initial launch, after which we'll downscale. The server has a Xeon E5-2670 8 core processor @ 2.6ghz and 30gb of RAM.

MySQL has been given tonnes of memory, and I've confirmed that MySQL is actually using that memory through "top":

innodb_buffer_pool_size = 15G

PHP has been given tonnes of memory (almost certainly too much now that I notice the comment saying "per script"):

memory_limit = 6000M

The Apache config is the out of the box default:

StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000

I've been running JMeter testing (basically "throw as many threads as possible at the site and request pages as fast as possible") and I can see performance dropping as the number of concurrent users increases:

• 5 users: ~1s avg response time
• 10 users: ~1.3s avg response time
• 20 users: ~2.5s avg response time

However, load on the machine is still very low. During testing "top" reports 26gb of memory free, and the load average from "uptime" doesn't go above 0.63. My suspicion is that we still have a lot of resources we could be throwing at performance, but Apache isn't making full use of those resources. That said, I'm a developer and not a sysadmin, and I'm not an expert with server performance.

What is the best improvement I can make to improve performance?

Solution 1:

Drupal Answers may be a better fit for this question, but here's a few things for you:

• Use caching in Drupal AND on the server as much as possible.
• IF most of your traffic will be anonymous, consider using a reverse proxy such as Varnish. There's a module that helps Drupal take better advantage of Varnish. Use it if you decide to implement Varnish.
• Look into using something like memcache as well.
• If your Drupal site is built in Drupal 6, then consider converting your raw Drupal code into a Pressflow Drupal site (it's easy). Pressflow incorporates a lot of performance optimizations (but its a moot point if you're running Drupal 7).

Now for Apache.... don't use an out-of-the-box Apache config. You need to tune Apache to your site. Assuming you're using mod_prefork (I'd say that's fairly likely, but only you can determine that) Here's the basics of how its done, but you should really hire someone who knows what they're doing.

• Figure out the maximum amount of memory you want Apache to be able to use.
• Heavily test your website, and determine how much memory each Apache process uses (using top).
• Take the Apache process in top that uses the most memory, add a little bit to it for good measure, and then divide your first number (maximum amount of memory you want Apache to use) by this new number.
• The number you get should be your MaxClients & ServerLimit variables.

This is certainly not the end-all answer. Tuning your server takes time and requires experience to get just right. Good luck.

Solution 2:

There's a few modules out there (Boost, Varnish etc.) which can help you boost things if you mostly have anonymous users. However, they won't help you that much if users create profiles on your site and log in.

This is how I set up servers for Drupal:

• Disable the Update Manager module. It slows down your site.
• I've replaced Apache with Nginx and phpfpm. You probably need to set up and configure this yourself. Even though I have not done any "formal" benchmarks, my impression is that Nginx + phpfpm serves requests faster. Here's a configuration for Nginx and Drupal.
• Optimise MySQL and InnoDB tables. See the MySQL Performance Blog.

There might be a lot small improvements which can take things further, but I think this could be at least a starting point for things to look at.