Software http load balancer?
Right now we only have one back-end server per site/web service. I'd be interested to hear people's experiences with various load balancer apps (something that runs on Linux).
What would you recommend?
Solution 1:
According to the episode 27 of the StackOverflow podcast, the Reddit guys really seem to like HAProxy. Here's a brief tutorial showing HAProxy being used with Amazon's EC2. You can find much more information at the HAProxy web site.
Solution 2:
The question doesn't provide specific information about load, features needed etc, so as such any answer is at best a guess.
Pound is a good choice for smaller / midsized sites. It offers HTTPS support and easy setup.
HAproxy can scale to saturate 10G Ethernet pipes, and offers connection limiting, i.e. sending only the number of simultaneous requests to each backend server as you have Apache children / Ruby on Rails instances to handle.
nginx is great as a allround load balancer and static file server. It can perform HTTP compression, URL rewriting and static file serving while doing load balancing.
Apache is in the 2.2 series a pretty good load balancer as well. Can do much the same as nginx, but places a higher load on the server than nginx. Very worth looking into if you're already familiar with Apache, and very mature.
Perlbal offers easy connection limiting to backend hosts, multiplexing requests across persistent HTTP connections, and easy setup if you're already using Perl.
Varnish cache is a reverse HTTP proxy, with basic load balancing support. It's not a great load balancer, but in some situations its in-memory caching of most requested objects can remove a high percentage of the backend server hits, and it has great performance. ESI includes are potentially interesting.
I'm not 100% sure, but off the top of my head Perlbal and nginx offer some support for changing the config without re-starting the load balancer. For larger sites this is critical, and it's one of the things that good commercial load balancer appliances do well.
All of the above are HTTP level (layer 7) load balancers. TCP/IP level load balancers have the potential to reach higher throughput, but are limited in other ways. The HAProxy author has written a good, readable overview of load balancing methods and issues: http://1wt.eu/articles/2006_lb/
LVS is a widely used TCP/IP level load balancer. Most firewalls can also do basic load balancing on IP level, by hashing the incoming request out over a range of backend IP addresses -- at least OpenBSD's PF and Cisco ASA and Juniper Netscreens can do this.