Scalable Web Application Hardware Topology Best Practice

Solution 1:

Scalability extends beyond the hardware layer and well into the application-layer. The ability of an environment to grow to bigbiglots is dependent to a great degree in the ability of the software at each layer to handle fault and maintain a coherent state across the entire environment. If the database driving your entire environment can't be sharded, you have introduced a scalability stopper by using that database. That kind of thing.

Amazon has some white-papers on developing for their cloud, and several of them are generally applicable to any scalable infrastructure.

http://aws.amazon.com/whitepapers/

There are some high-level principles you need to keep in mind when scaling:

• It must survive the fault of any single component, and do so without human intervention.
• It should respond dynamically to high loads, without human intervention.

Don't just use a load-balancer, use a trio of load-balancers that present as a single LB so that if any one fails, the others can pick up the load.

The web/app servers should have user-state available on all nodes, so if a user is pushed to another web/app server their session is preserved. Best case, all state can be served from all servers.

There should be redundant routers in your network, so you can take one down without stopping all traffic. HSRP is one protocol to enable this.

Your database-plan must include how far you will scale the one DB server before you start sharding, and start developing with sharding in mind once you get close to that point.

Caching layers (memcached) may be needed in your environment for performance reasons.

Once you get large enough, you need to plan on how you'll host your environment from multiple separate locations (such as US West and US East, or US West and Europe) via Anycast or GeoIP. Moving data between the separate locations will be a challenge, and you need to start developing against that assumption once you get close to needing separate locations.

Ruby itself has some scaling issues, focusing on its ability (or not) to take advantage of multiple processors on servers. The capabilities are there, but rather new so not well understood in the dev community yet (or so I gather). As Ruby matures, some of these issues will go away.