Nginx rate limiting - only slowdown if ceiling hit

We recently switched to nginx and I was happy to see that rate limiting seems easy enough there. I set up the following limit for a particular zone:

limit_req_zone $binary_remote_addr zone=PHPUtilities:10m rate=60r/m;

I applied that zone to a location block with a burst of 20. That all seemed fine, but what I was surprised to see was that I would quickly get slowdowns where the server would refuse to respond for 5 seconds or so. It seems like nginx takes the 60 requests per minute and divides that down to 1 request per second. So if we loaded several scripts simultaneously, nginx would serve the first request right away and then delay the others.

I'd rather it behaved sort of like "ok they get a free 60 requests per minute. Only once they hit 60 will we begin limiting them, first by slowing down the next 20 requests (the burst). Everything after that will crap out until they stop requesting so much, or fall below the 60 per minute limit."

Is there a way to do that?

PS. I accidentally asked this question on Stack Overflow first, then realized it was more appropriate here. :(

Solution 1:

I think you want to use the nodelay parameter as well as burst:

location /login/ {
    limit_req_zone $binary_remote_addr zone=PHPUtilities:10m rate=60r/m burst=20 nodelay;

    proxy_pass http://my_upstream;
}

With the nodelay parameter, NGINX still allocates slots in the queue according to the burst parameter and imposes the configured rate limit, but not by spacing out the forwarding of queued requests. Instead, when a request arrives “too soon”, NGINX forwards it immediately as long as there is a slot available for it in the queue. It marks that slot as “taken” and does not free it for use by another request until the appropriate time has passed (in our example, after 100 milliseconds).

From https://www.nginx.com/blog/rate-limiting-nginx/.