How to scale up a web server supporting long polling

Solution 1:

I don't think your 60k clients are the actual problem. You will more likely have problem with not enough file descriptors, but that should be easy to fix as part of OS configuration.

Here's why connections will not be your problem. Each connection is characterised by its source ip address, source port, destination ip address and destination port. Inside the network stack this quadruple is used to match packets to file descriptors(each file descriptor represents a connection). Your server has fixed destination ip address and destination port (your server is destination for their client) but source ip address and source port are variable. Port is a 16bit number therefore maximum number of connections from one client is 64K. IPv4 address is a 32 bit number which gives you 4,294,967,296 possible source addresses. Doing some basic maths, your server could have 64K * 4,294,967,296 connections mapped to a single source ip and port.

This is why you will more likely have problem with maximum number of open file descriptors then the number of clients.

Solution 2:

The most simple approach might be to implement load balancing at the DNS level.

Means: have a round robin DNS entry that balances to 2, 3, or more physical loadbalancers.