nginx automatic failover load balancing

I'm using nginx and NginxHttpUpstreamModule for loadbalancing. My config is very simple:

upstream lb {
    server 127.0.0.1:8081;
    server 127.0.0.1:8082;
}

server {
    listen  89;
    server_name localhost;

    location / {
            proxy_pass      http://lb;
            proxy_redirect  off;
            proxy_set_header        Host            $host;
            proxy_set_header        X-Real-IP       $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

But with this config, when one of 2 backend server is down, nginx still routes request to it and it results in timeout half of the time :(

Is there any solution to make nginx to automatically route the request to another server when it detects a downed server.

Thank you.

Solution 1:

I think that it's because nginx is not detecting that the upstream is down because it's on the same machine.

The options that you're looking for are: proxy_next_upstream and proxy_connect_timeout.

Try this:

location / {
        proxy_pass              http://lb;
        proxy_redirect          off;
        proxy_next_upstream     error timeout invalid_header http_500;
        proxy_connect_timeout   2;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
}

Solution 2:

Hey, please see the wiki: http://wiki.nginx.org/NginxHttpUpstreamModule#server

Basically if a failure is detected the backend will be marked as down for x seconds and it will try again. So if you keep seeing connections it's probably nginx that keeps checking if the backend has become available.

It should, however, try the next entry in the upstream block, so you shouldn't actually see that no backends are available if only one is down.