nginx denies all requests to a virtual host . requests are coming from a nginx tcp forwarder

Solution 1:

This is happening because your nginx load balancer makes a new TCP connection to your nginx web servers, causing the original client IP address to be lost.

You can solve this using the PROXY protocol. This protocol sends the original IP address information when the new connection is opened so that your web servers can be aware of it.

To set it up, you need to make the following changes:

• On the load balancer, set proxy_protocol on;:

  stream {
      server {
          proxy_pass somewhere;
          proxy_protocol on;
  

• On the web server, accept the PROXY protocol, and enable the Real IP functionality for the load balancer's IP address.

  server {
      listen 443 ssl http2 proxy_protocol;
      listen [::]:443 ssl http2 proxy_protocol;
      real_ip_header proxy_protocol;
      set_real_ip_from <load balancer's IP>;
  

There are additional tweaks you can make to the PROXY protocol which you can find in the nginx documentation but this should get you started and solve the immediate problem.

Solution 2:

My approach would be:

Filter in the load balancer using SNI:

Basically: SNI allows the proxy to see the domain name, without terminating TLS. Here is a guide. And here are the nginx docs to combine it into a solution. If you can do this, you can then deny based upon $ssl_preread_server_name.

if ( $ssl_preread_server_name ~* mysite1.example.com && $remote_addr !~* <whitelisted_ip> ) {
           return 404;

The real_ip module also seems like a good solution.

Edit

The PROXY protocol would actually be better than real_ip. But this would still mean, that your backend servers get hit by requests that they'd end up blocking. The SNI solution catches them at the load-balancer.