When to Add another server(s) [duplicate]
When is the right time to start adding(or thinking about adding) servers to your Web Application? ...(going from a single server ... to multiple)
Think about it: From the beginning.
Start adding: When you get your first "server too busy" error. Anything sooner is premature optimization.
(Unless your web app is mission critical, in which case you probably aren't starting from scratch and don't need to poll the serverfault.com community.)
But seriously, for modern consumer web applications, getting "server too busy" can actually be a good thing. It certainly never hurt Facebook, Twitter, or YouTube. The danger with adding servers too early is that your app will never be as popular as you are expecting and you've ended up wasting money that could have been spent on feature development.
If you're one of the lucky few who actually has a hit web property, then (a) congratulations, and (b) you'll be able to measure your average response times from your log files and can take a more metric-driven approach to adding servers.
Personally, I always start with a separate web server from the DB server.
(based on IIS6 and SQL Server)
- A database will just chew up resources (RAM, DISK, CPU) and compete with the web server. This can make the web server appear unresponsive and the database "bad"
- Security. Not a good idea to have more on the "edge" than you need. Move the DB to inside your DMZ and poke a non-standard hole (not 1433) in your firewall for SQL traffic.
- Separation of roles. Now you can see if it's the DB or the IIS or application that's chewing up CPU
However, we do ALL this on a vmware server (3 node cluster in fact), and the webserver and the db server only have 1 vCPU and 1GB RAM. And they support a fairly busy suite of websites (external and intranet) without too much sweat.
But I know if I get a surge of popularity (and it might happen one day!) I can quickly reconfigure the vm's with more RAM and CPU.
On a practical note: Monitor your performance over time. Track changes and be proactive.
When is the right time to start adding(or thinking about adding) servers to your Web Application?
When you are designing your application. I've seen too many applications not designed to use multiple servers, and reverse engineering that later can be horrendous.
Make sure you test on multiple servers, too. Again. I've seen many applications work fine in dev/test only to fail in production because they couldn't deal with load balancers, or firewalls, multicasting hadn't been considered etc, etc.
And the time to add another server is when your capacity management statistics suggest that you will run out of capacity in just over the time it will take you to add another server.
You are collecting cpacity statistics? No? Then another thing to talk about with your application developers and infrastructure management people.
I disagree that waiting until you actually get 'server too busy' and irritating your users is the correct thing to do. Adding a new server to a production environment can be a lengthy process, and waiting until you have errors before you start is not really wise.
As others have mentioned, the question is vague, but there's really a pretty simple answer to it:
You need to add another server when limitations on the current one start costing you money.
If you're offing a free service, then there's really no incentive to add additional servers.
If you're making money off your service, then you can do a cost/benefit analysis: how will additional servers improve performance, and how much more profit will result?
One of the things you should do is load test your existing one-server application to see how many sessions and users it can support at once. Once you know that, you can monitor those same metrics over time. When your numbers begin to approach the known limits, it's time to start preparing new hardware for deployment. You will need to consider the lead time for additional servers so that you can deploy before the existing system reaches its limit. This is a very general overview, and the details are covered very well in the book "The Art of Capacity Planning" by John Allspaw.
Having said that, I would break the database server and web server into their own systems as a starting point, then load test each to determine the theoretical load. Usually the web server will become the bottleneck first, so your application should be written with scalability in mind. The question mentions sessions, and depending on the platform you're using there are several ways to share sessions between servers, store them in the central database, or use load balancing hardware that will direct the same user to the same server unless it becomes unavailable for some reason.
But to answer the core question "when do we know when to do it," that will be all about the metrics you're collecting and comparing them against known limits arrived at through load testing.