websockets , loadbalancers and 64k ports

with the websockets and persistent TCP connections how are load balancers going to cope with the 64k port limit if they are handling a large farm of servers in the backend? need. someideas on setting up the infra for an app that can. potentially have 100k connections.

Your question appears to assume a SNAT (aka NAPT) translating load balancer. Here's some ideas about solving the 64k ephemeral ports problem. My experience is with F5 Networks' BIG-IP product (so the links are to their site), but the concepts are the same for other vendors:

• Don't SNAT. If source ports are not translated, there will be no 64k limit. To turn off SNAT, you need to have the inside address of the load balancer set as the route (usually default route) on the inside servers.

• Use a SNAT Pool. This makes a pool of internal IP addresses available to the load balancer to translate to. For example, two IP addresses in a SNAT pool will give you 128k ephemeral ports so 128k simultaneous TCP connections.

More advanced approaches:

• Use "n-Path Routing" (that's F5's term, others may call it "Direct Server Return"). This doesn't translate the client address or port (or destination IP, for that matter!), so also makes the ephemeral port issue go away. The responses from the servers bypass the load-balancer. The way you achieve this is with loopback adapters hosting the same IP on all your servers, so that they'll accept the traffic.

I should point out that Websockets are a special challenge for traditional HTTP load balancers, as the connections live much longer - people do run into the ephemeral ports problem when they may never have before. In my view, the best solution is one that removes the SNAT requirement (first or third solutions above). Scaling is much improved, and the load on the load-balancer is reduced. The added complexity is worth it.

Here is a good article on the issue, from F5's Lori MacVittie: HTML5 Web Sockets Changes the Scalability Game

Keep in mind that a socket is a tuple of sec/dst address, src/dst ports and protocol and, as such, the number of permutations is a lot more than 64k. There are some situations where outbound connections on proxy servers might have issues based on particular implementations but port numbering hasn't been a big issue traditionally.

I found the most detailed answer to my question on StackOverflow.