What you want is generally called a load balancer. Load balancers can either come in the form of an appliance, F5 Big-IP boxes for instance, or in the form of a load balancing application running on top of commodity hardware. One example of this is HAProxy, which coincidentally, is currently doing the load balancing for all of the StackExchange sites.


If you don't want load balancing, all you need is a simple failover clustering solution which sends heartbeats between the cluster members and assigns ownership of the virtual IP that you'll be using to access the service. There's dozens of them out there.

On Windows, the de facto answer is Microsoft Clustering Services (MSCS), which is included with Enterprise and Datacenter editions. On Linux/Unix, you probably want to be looking into Heartbeat (simple, but not super-configurable), Pacemaker (robust, but a bit more complex), or another similar system.


There are some opensource solutions here that will help.

The first one I would check out would be PFSENSE

pfSense will run on virtually any hardware - as well as in a vps. the system will allow for load balancing, clustering, as well as a variety of other services

pf uses the idea of the "carp" method for load balancing and fail over- very easy to setup.