Why is Apache spawning so many processes?

Solution 1:

Shamely, most hosting solutions out there people use, decide to parse php content through Apache with mod_php.

Your PHP installation by default will surely not be thread-safe, this effectively forces Apache to use a non-threading mpm (multi-process module).

Your Apache installation uses prefork (as you can see in the apachectl -V output). This mpm spawns a process per request. So if you are under a bit of load and nowadays knowing how many concurrent requests some browsers send, that list of process is quite usual.

All this make apache crawl and suffer from the load your php scripts cause, making Apache look as it is not performing well, when it is all due to php parsing and unperformant php scripts.

Now, what should you do to avoid this?

Relief Apache from the burden of mod_php, move your php parsing to php-fpm, that will allow you to use Apache HTTPD's "event" mpm, that would even allow you to have 1 process with a thousand threads if you want, and not only that, it is quite more responsive to load spikes and will be as cool and fast as any other http server out there. And most importantly, Apache processes won't be stuck in the hundreds.

You can find some tips on how to configure Apache for this at: Apache Official Wiki PHP entry and mod_proxy_fcgi

Solution 2:

It's clear that even 40 is too many Apache processes. You have only 1GB RAM and 2GB swap, and you're using all the RAM and more than half of the swap. The reason your processes are in D status is because your virtual machine is thrashing in and out of swap. It's unlikely to recover on its own; you may as well reboot it.

Decrease the MaxClients number significantly. For a low traffic site on a micro instance, I can't imagine any way you would need it to be higher than 10 (nor for the instance to be able to handle much more). In fact, the settings you have commented out look like a very good starting point, and you should probably restore them.

Once you've recovered, you can start looking at what else on the system might be eating up lots of RAM.