apache webserver unresponsive with server-status showing all child processes waiting for connection [closed]

You should enable extended status of mod_status (http://httpd.apache.org/docs/2.2/mod/mod_status.html#extendedstatus) in order to monitor the current hosts and requests being processed. I think there's a script(s)/page(s) which takes too much time to release the connection and it makes the connections stacking.

First: Check your Max open files limit on the process. An active socket connection counts as an open file. cat /proc/###/limits is a good way to check the effective value for another process. You can get a list of open files with lsof -p ### where ### is your web server's process id. You can compare lsof -p ### | wc -l to see how close you are getting to the limit. You should also see messages in apache's error_log if you are hitting the limit.

You need a file handle for each socket connection, and also for each cgi script or data file reference. For 920 MaxClients, you should configure at least 4,000 files for the httpd process. You can increase the number of files by adding a file in /etc/security/limits.d/ with the following contents. Ensure the user name matches what you are using for your web server.

apache soft nofile 10000
apache hard nofile 10000

Second: If port exhaustion is your problem, you can adjust the some ip settings in /etc/sysctl.conf. (Starting with net.ipv4.tcp_fin_timeout). This is usually a problem only with lots of very small connections. Many TIME_WAIT sockets is one indicator of this, but this indicates port exhaustion only when accompanied by errors in syslog about possible SYN flooding and Sending cookies. You should also ensure your server is behind a firewall that can thwart malicious SYN attacks.