MPM Prefork, too many apache2 process?

I gave you the answer to this in the comments over on Server not responding to SSH and HTTP but ping works, but apparently you don't believe me. Really, it's true!

You need to size MaxClients / ServerLimit to your system. The "5-10 settings for Min/Max Servers" which you mention are basically irrelevant — that's just the number of extra servers hanging around not doing anything that Apache will retain.

In order to set MaxClients appropriately, look at the typical high-water mark for your httpd (or apache2) processes, and then divide your available memory by that. Best to drop down by a little bit to give the rest of the system room to breathe. Since you've got 4GB of RAM, and 185MB processes, that means your ServerLimit value should be 21 at most — probably 20 or 19.

Now, it may be that 190MB is atypical. You can set the ServerLimit higher, based on a different estimate of typical usage, but then you're basically gambling that you'll never have a spike. If it does happen, your system will be out of memory.

If you can find a way to constrain your per-worker memory usage, that's gonna be a win. I'm betting this is a case of PHP Ate My RAM. Can you code your app to live within a lower memory_limit? If you can't do that, you need a different model under which to run your PHP. If you can't do that, you need to buy more RAM.

Apache's prefork MPM self-manages servers. It will always start with StartServers daemons, and will never run fewer than MinSpareServers once it gets going. It will also eventually retire/kill off servers in excess of MaxSpareServers if they're idle long enough (I don't recall what "Long Enough" is in this context, nor if/how it can be modified).

ServerLimit sets the maximum number of apache daemons that can be running at any given time -- This is why in your situation you can have hundreds of sleeping apache processes (they got spawned to service a flood of requests and haven't been idle long enough to be killed by the mother process yet).

Personally I think 1250 is a pretty high value for ServerLimit/MaxClients -- 250 may be a more reasonable number (though this may result in the occasional 503/Server Busy error if you get a massive flood of requests: if that becomes a chronic issue you can increase the number or add more servers to handle the load).

Relating this question to your previous one Re: an out-of-memory crash, definitely follow the guidance from the Apache Manual on this parameter:

Most important is that MaxClients be big enough to handle as many simultaneous
requests as you expect to receive, but small enough to assure that there is enough
physical RAM for all processes.

…and my personal axiom: It's better to give a client a 503 page than knock the server down. :)

Turn off Keepalives and set MaxClients to 150. The most likely reason you have 260 processes just sitting there is because Apache is dutifully holding browser connections open because KeepAlive on is set in you apache config file.

Calculate how many servers you can have running within the constrains of your system's RAM by running this command:

$ ps -ylC apache2 | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Process Size (MB): "x/((y-1)*1024)}'

If will produce output like:

Apache Memory Usage (MB): 1608.76
Average Process Size (MB): 55.4745

Now stop apache and find out how much RAM you have available without it by using free:

               total       used       free     shared    buffers     cached
  Mem:       7629384    7415780     213604          0     333428    5341884
  -/+ buffers/cache:    1740468    5888916
  Swap:      7629380       7968    7621412

(above is in kilobytes. free -m would show you megabytes.)

Linux will fill available memory with buffers and cache, so adding free+buffers+cache (213604+333428+5341884) yields 5888916 Kbytes available.

588916K available / 55474K per apache process = 106 servers. But set it lower than that to leave some breathing room.