Slow http responses

I am running an apache web server on debian 6.0 with kernel 2.6.32-5-amd64.

In certain times of the day, usually when more people are online, the server have very slow responses. I am sure, that is not load, it is not DB, it is not PHP, accualy this command:

time wget 127.0.0.1

is quite slow:

--2013-09-20 15:36:49--  http://127.0.0.1/
Connecting to 127.0.0.1:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2013-09-20 15:36:52 ERROR 404: Not Found.


real    0m2.663s
user    0m0.004s
sys     0m0.004s

Response time is up to 5sec. The point of waiting is after "HTTP request sent, awaiting response..." Page generating after this lag common for all http requests is fast, like 50ms.

I tested my web using http://tools.pingdom.com/fpt/, first response is always slow, most of content load is also slow but few are quite fast, but I cannot tell, if this tool have some sort of cache.

telnet 127.0.0.1 80

is fast, the response show without delay:

Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Where can be the problem? It must be something before the request goes to apache.

Please tell me at least about some debugging tools for this.

Solution 1:

I don't think 2.6 seconds is terribly slow for your average site that wasn't created by a team of full time engineers (See: google.com).

Your problem is probably in I/O, either network or disk or memory or etc.

You're going to have to trace through the application stack. This involved looking into your web server software (apache httpd, ngnix, etc), the php stack, the php code, and any backend storage mechanisms like File systems and MySQL that support the stack.

The delay could be caused by a slow disk, an unindexed mysql query, a bottleneck at the network, or even an optimized loop or function in the code. I don't think we will be able to help you much in this unless you can provide details about your environment and ask specific questions.