nginx proxy_cache: limit parallel requests to a backend
Solution 1:
Take a look at the limit_conn module. While all the examples I've found have been on limiting by remote IP, if you pass a constant to limit_conn_zone instead of the remote IP you will be limiting by total connections to the server.
Something like this:
upstream myservice {
server 127.0.0.1:80;
server 123.123.123.123:80;
}
set $const 1;
limit_conn_zone $const zone=one:1m;
location / {
limit_conn one 10;
proxy_pass http://myservice;
proxy_set_header Host myservice;
}
Edit: It seems like this module is only available in more recent versions of nginx (possibly >=v1.1.8). You may need to compile from source if you want to use this option and have an older version of nginx.
Edit 2: If your nginx is only doing proxy duties, take a look at worker_connections
Solution 2:
I think the configuration options you require are:
proxy_cache_use_stale updating;
proxy_cache_lock on;
This serves out of date elements from cache, and issuing updates from backend. So when your backend servers a page after 2 seconds, the client already got the old ressource from cache.
The second locks one element from cache, while already a update-cache request to backend is running, so there are no paralell requests for one cache-item.
This should be much better then hard proxy limits.
If you still want hard proxy limits for connections, you can run a nginx in backend and rate-limit connections or requests there. Else you might want to try a connection limit with iptables, which should perform much better than in nginx.