Does having a load balancer allow you to re-use socket connections?

Solution 1:

I'm assuming you are refering to SO_REUSEADDR rather than connection multiplexing. The only benefit this gives is that a socket in TIME_WAIT does not prevent a new socket binding to the same address. I've never come across a system which actually uses the TIME_WAIT phase for what it is intended for - if you've got lots of sockets in TIME_WAIT maybe you should just reduce the timeout.

Making it all go faster would help too - but you've not given a lot of information on the set up here and the pattern of traffic.

Any type of proxy load balancer will give you even fewer clien connections than you have now (since each incoming connection requires a connection to the back end devices). OTOH RRDNS will halve the load without the complexity and cost of a SPOF.

it is a single POST request

So we talking HTTP here? If so then then there's ENORMOUS scope for tuning.

Solution 2:

All leading load-balancers have a form of connection reuse and pooling, they just call it something different. They are ultimately designed to process packets very quickly and efficiently.

F5 calls it OneConnect, Netscaler TCP multiplexing…..I don't know enough about the other vendors or open-source solutions.

This feature enables load-balancers to re-use TCP connections on the server-side and greatly improves the performance of the web application on the back-end. When a connection is finished so-to-speak, its moved into a reuse pool on the load-balancer and matched to a subsequent connection if suitable (if it's not suitable the load-balancer will open a new TCP connection to the server). There's no connection teardown and so there's no need for the subsequent three-way handshakes on new connections or for the server to assign or remove the memory assigned to the open sockets. The other thing to note is that the server-side connections are sequential, it's not pipelining.

Without connection re-use, it's a 1-1 correlation (between client and server) and in my experience, performance (of the application) is drastically reduced with load on the back-end server much higher. If it's SSL traffic, you'll see a CPU hit due to the key exchanges.