nginx refresh upstream server IP

I have a Docker setup with one Django container and one nginx serving static files. I have nginx configured in the standard way:

upstream main_web {
    server web:8000;
}
server {
    location / {
        proxy_pass http://main_web;
        #...
    }

}

I use the /etc/hosts entry to get the IP address of the Django container.

When I restart the Django container, its IP address is updated, which reflects to the hosts file. But nginx keeps giving a 502 Bad Gateway error. Things work fine when I manually restart nginx.

Isn't there a way to tell nginx to resolve the IP again if it's not reachable?

Solution 1:

When you use a variable to specify the domain name in the proxy_pass directive, NGINX re‑resolves the domain name when its TTL expires. You must include the resolver directive to explicitly specify the name server (NGINX does not refer to /etc/resolv.conf). More here

Using Docker, you can check your current dns resolver and add that to your nginx configuration. Check /etc/resolv.conf.

For example using docker-compose you can set the name of the service in your nginx vhost configuration. Asume we have this setup:

docker-compose.yml

version: "3"
services:
  my-backend-service: # we will use this name in our nginx vhost conf
    image: some-image
  lb:
    image: nginx
    volumes:
      - ".docker/etc/nginx/default.conf:/etc/nginx/conf.d/default.conf"
    ports:
      - 80:80

A few things about the above configuration, note that I do not add a container-name to the backend service, so I can scale it freely. Also pay attention to the service name, we will use that in Nginx vhost conf so it resolves to the instances ip via DNS. Finally, Nginx is listening on port 80 and the backend-services are listening on 8080 (as an example, could be anything)

default.conf

resolver 127.0.0.11 valid=10s; # 127.0.0.1 comes from /etc/resolv.conf on the nginx container

server {
    location / {
        set $backend_servers my-backend-service;
        proxy_pass http://$backend_servers:8080;
    }
}

To test this:

docker-compose up -d
docker-compose scale my-backend-service=5
docker-compose logs -f my-backend-service

Now you can start making requests to localhost and Nginx is going to use the DNS resolver you specify to get the ip addresses of the backend services, and will re-resolve the domain name when the TTL expires.

nginx refresh upstream server IP

Solution 1:

Related

Recent Posts