google cloud http(s) load balancer returning 502 despite healthy backend service

I've got a GCE http(s) load balancer configured to direct traffic to 2 backends: - the default backend - a second backend with a separate host/path rule (<host>:/*)

Both backends exist on the same instance group - a container cluster running two kubernetes services that expose different NodePort.

The instance group has two named ports, one for each of the exposed NodePorts.

Despite both backends are reporting healthy, all requests to the second backend return 502.

The services themselves are not the problem - if I switch the service behind the 2nd backend to be default, it works fine.

NOTE: I've updated/simplified this description considerably since it was first posted, as I'm able to reproduce the problem with a much simpler configuration.

Solution 1:

Traffic from the load balancer to your instance is not enabled by default. Unfortunately this is not well documented, and really, when you create a load balancer this should happen automatically.

Try adding this firewall rule the network that your load balancer and VMs are on:

130.211.0.0/22   tcp:1-5000   Apply to all targets