Nginx as reverse proxy: how to properly configure gateway timeout?

We have configured Nginx as a reverse proxy to an Apache server farm, but I'm running into trouble with the gateway timeouts.

Our Goal in human readable form is: "Deliver a request within one second, but if it really takes longer, deliver anyway", which for me translates into "Try the first Apache server in upstream for max 500ms. If we get a timeout / an error, try the next one and so on until we finally succeed."

Now our relevant configuration is this:

location @proxy {
    proxy_pass         http://apache$request_uri;

    proxy_connect_timeout 1s;
    proxy_read_timeout 2s;

}

[...]

upstream apache {
 server 127.0.0.1:8001          max_fails=1 fail_timeout=10s;
 server 10.1.x.x:8001           max_fails=1 fail_timeout=10s backup;
 server 10.1.x.x:8001           max_fails=1 fail_timeout=10s backup;
 server 10.1.x.x:8001           max_fails=1 fail_timeout=10s backup;
}

The problem here is that nginx seems to misunderstand this as "Try to get a response from the whole upstream cluster within one second and deliver a 50X error if we don't - without any limit on how long to try any upstream server", which is obviously not what we had in mind.

Is there any way to get nginx to do what we want?

I think what you need is:

max_fails=0

and

proxy_next_upstream = timeout

per the docs:

max_fails=number

sets the number of unsuccessful attempts to communicate with the server that should happen in the duration set by the fail_timeout parameter to consider the server unavailable for a duration also set by the fail_timeout parameter. By default, the number of unsuccessful attempts is set to 1. The zero value disables the accounting of attempts. What is considered an unsuccessful attempt is defined by the proxy_next_upstream, fastcgi_next_upstream, uwsgi_next_upstream, scgi_next_upstream, and memcached_next_upstream directives.

http://nginx.org/en/docs/http/ngx_http_upstream_module.html

and:

Syntax: proxy_next_upstream error | timeout | invalid_header | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | non_idempotent | off ...; Default: proxy_next_upstream error timeout;

Context: http, server, location Specifies in which cases a request should be passed to the next server:

error

an error occurred while establishing a connection with the server, passing a request to it, or reading the response header;

timeout

a timeout has occurred while establishing a connection with the server, passing a request to it, or reading the response header;

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream