More RAM vs. more servers

I was recently asked "Do you know when to decide between going for more RAM or more servers?" (in the context of scaling data mining applications).

I had no idea, so what are some ways to decide? I have very little knowledge of architecture and scaling (my understanding of computer memory and what a server does is limited to the high-level basics), so tips on learning more about these things in general are also very welcome.

"Do you know when to decide between going for more RAM or more servers?" (in the context of scaling data mining applications).

The answer is as soon as you give me the metrics for the server in question, I'll tell you which (or if its worthwhile at all to add either). This type of tuning isn't voodoo (unles you are using applications without instrumentation and server OSes without instrumentation- then yeah it's voodoo) it's science. Measure the application and the server. In a nutshell using the monitoring metrics figure out where the performance bottleneck is and add more of it.

There is generally quite a bit of voodoo (or at least trial and error) in server/application performance improvement.

The general rule for the specific question asked is to first increase the memory until it can't be increased any more OR until more memory no longer improves performance. With memory relatively cheap, it may be more straight-forward to simply max the memory. Also, if the application is disk-heavy, upgrading to high speed drives or high-performance controllers can make a difference.

However, the very general nature of the question leads me to think that there hasn't been other attempts to improve performance. I agree that hardware is cheap, so even throwing more servers at a problem is easy enough to accomplish. But, I would also make sure that other avenues, particularly tuning of the OS and the database, have been done. Sometimes small tweaks to the database, the OS, or even the application configuration can result in huge performance improvements.

Search on this site with your specific OS, database and application and you might well strike gold.

As an enterprise architect I've dealt with this issue on an almost daily basis. Vertical or horizontal scaling?

What are your needs?

Do you need to support more users? Do you need to improve the speed of the service? Do you need both? Do you need high availability 99.9999 or can your users take downtime?

To begin you need to capture performance metrics on the current system. Number of active users, RAM and CPU loads, Disc I/O - find out where your bottlenecks are.

Possible solutions based on issues: Start with optimization of current resources. If your application is database driven optimize the database with query and thread caches, indexes, etc. If your sharing a server with other applications explore moving to a dedicated server. (Look into virtualization for less active / critical applications to free up dedicated resources).

current machines are at full capacity, RAM and CPU heavily loaded, high disc I/O - calculate cost to add RAM, can you switch to a faster disc I/O (RAID, SATA in place of ATA)?

If you need high availability then you probably need to add hardware and load balancing anyway.

Is it cheaper to add hardware upgrades or add new servers? Which fits the long range goals and growth?

When is your IT dept's best time to spend money? Do you have funds now or do you want to shift the expenses to another quarter / year? If funds are an issue then optimize now or explore freeing up hardware from other applications to add a temp load balancing solution.

Don't be afraid to explore numerous solutions. Vendors might want you to buy a load balanced, SAN storage centric solution where a new server with iSCSI RAID 10 on board will work for 10 percent of the cost.

If your CPU is still heavily loaded after optimization then you need to add / replace hardware. If your disc I/O is the bottleneck and you can't upgrade storage technology then you need to replace hardware or add network storage / attached storage solutions.

Capture performance metrics. Optimize, improve and capture metrics again. Keep documenting performance increases / decreases so you can turn in a report that documents how much you spent and how much performance gain resulted. These are the type of possible success stories that make admins into architects, architects into project managers and PMs into upper management when done right.