Multiple vCPUs or multiple identical VMs, which is better?

Solution 1:

Can VMware ESXi handle multiple vCPUs better than VMware Server 1 could over three years ago?

Yes, they're totally seperate product lines, and ESXi can handle multiple vCPUs very well. I'm running a 5 node vmware farm right now with a mix of machines on each box, each with 1, 2 and in some cases 4 processors as needed and it works very well indeed.

Which will perform better, a single large webserver VM with 2 - 4 vCPUs, or 4 VMs with a single vCPU, in a load-balanced configuration?

That's less of a VMWare question and more of a application/website testing question. VMWare (or any other modern bare metal virtual machine) can do either way - there might be a cost to having lots of vCPUs attached to one guest but this might be offset by the resource costs of having multiple guests running and by performance issues (or improvements, for that matter) your web application might realise from a more distributed approach.

On that last issue, its really a matter of testing and architectural preferences rather than the one true way, but my general feeling is that if you can distribute the application over more than one virtual machine easily, its better to build a few smaller virtual machines than one big one. Gives you more flexibility and helps performance in the long run as you can move things about if you have to. In other words, scaling outwards rather than upwards

With physical machines I think we think of scaling upwards, putting as much into one OS "instance" as possible to get the most out of the cost of the licence and of the hardware, but with virtual machines those equations change a bit - if you can get more than one virtual OS instance for the price of a licence then this might change the way your server infrastructure is designed.

edit

http://blog.peacon.co.uk/understanding-the-vcpu/ is an article Iain mentioned while discussing this question in chat that explains in some detail why its important to think about vCPU allocation when creating your VMs.

Solution 2:

I can tackle this question in a number of ways, but I'll start of with horizontal vs. vertical scaling.

Horizonal scaling

Pros

• Can scale very large
• Can scale across hosts (ie a vmware cluster)
• Fault tolerant
• Low/No downtime during maintenance

Cons

• More overhead from VM OSes and Load Balancers
• More maintenance/management (eg patching, SSL certs)
• More website version management across web servers
• Additional testing and development may be required for web site behind load balancer
• Licensing?

Vertical Scaling

Pros

• Scales easier (just add another vCPU or two, or more memory)
• Only one web server to manage, license, etc.

Cons

• Downtime during maintenance/upgrades
• Scaling too big can cause performance issues in a VM.
• Doesn't truly scale.

So, both have their goods and bads, and I am a big fan of horizontal scaling, but here's what I'd recommend to you:

1. Scale vertically to a certain point (I'll get to that certain point in a minute).
2. Do your homework on what it is going to cost time and money wise to get going horizontally.
3. Give yourself plenty of time to test your horizontal scaling solution.
4. When you've reached the point where it's time to scale horizontally, you can ramp back down your vertically scaled server, make some clones, throw them behind your already configured load balancer, and watch it rock and roll.

Okay, so what is that certain vertical scaling point?

It depends on your hardware.

VMware has done a lot to vCPU scheduling since ESX 2.x, and there is more room to grow vertically. How much really depends on the CPUs you get.

Sizing VMs for performance nowadays requires understanding of NUMA (Non-Uniform Memory Access. Basically, you don't want a VM to be bigger than your CPUs NUMA node. Here are some links to help guide you in that area:

http://frankdenneman.nl/2010/02/sizing-vms-and-numa-nodes/

http://frankdenneman.nl/2010/09/esx-4-1-numa-scheduling/

http://frankdenneman.nl/2011/01/amd-magny-cours-and-esx/

I guess I'll wrap it up by saying that CPU has never been too much of an issue for me, unless you are oversubscribing CPU on your host. Instead, memory is normally the culprit. Ballooning/swapping memory os a very noticeable performance issue.

Good luck!