Limit max number of simulataneous connections from NLB to a specific instance

Solution 1:

Nope, that's not possible. You could try and trigger AutoScaling, but its not going to be instant. If you need that kind of custom logic you'd need to use a 3rd party solution to act as a load balancer, like HAProxy or NGINX