CPU Cores: The more the better?

NOTE: This answer was written 8 years ago. Parallel programming has since become more relevant. This is likely due to inherent clock speed limits and the fast approaching transistor size limits.

Your primary problem is software not written for multi-core.

Look at Jeff Atwood's excellent article on Choosing Dual core or Quad Core.

for most software, you hit a point of diminishing returns very rapidly after two cores. In Quad-Core Desktops and Diminishing Returns, I questioned how effectively today's software can really use even four CPU cores, much less the inevitable eight and sixteen CPU cores we'll see a few years from now.

You are answered here (highlight copied from Jeff's article),

However, there were some surprises in here, such as Excel 2007, and the Lost Planet "concurrent operations" setting. It's possible software engineering will eventually advance to the point that clock speed matters less than parallelism. Or eventually it might be irrelevant, if we don't get to make the choice between faster clock speeds and more CPU cores. But in the meantime, clock speed wins most of the time. More CPU cores isn't automatically better. Typical users will be better off with the fastest possible dual-core CPU they can afford.

The issue of the Front-Side Bus (that term always amused me).
With Nehalem things change... as ArsTechnica said last year.

Moore's Law has given processor designers an embarrassment of transistor riches, and nowhere is that more apparent than in Intel's 45nm Nehalem processor. Debuting in 4- and 8-core variants later this year, Nehalem packs a ton of hardware into a single processor socket. (Early numbers put the transistor count of a quad-core Nehalem at 781 million; no numbers for the 8-core model have appeared yet.) But trying to feed all of that hardware with the Intel platform's existing frontside bus architecture would be folly. So, just as importantly, Nehalem also sounds the long-overdue death knell for Intel's positively geriatric frontside bus architecture.

The radical change in Intel's system bandwidth situation that Intel's new QuickPath Interconnect (QPI) represents is perhaps the largest single factor that shaped Nehalem's design. Between QuickPath and Nehalem's integrated memory controller, a Nehalem processor will have access to an unprecedented amount of aggregate bandwidth, especially in two- and four-socket implementations.

AMD moved the memory controller into the processor earlier and used Hypertransport.

There is, in my experience, a huge performance benefit going from one to two cores. Suddenly one high CPU program doesn't lock up your machine or make it unusably slow. Massive difference.

But two to four? For 99% of people it won't make a difference. You will have to be running a lot of programs at once or be using programs that can take advantage of more than 2 cores (and realistically there aren't that many of those). Certain media encoders spring to mind.

There's also a difference between the Intel Core 2s and the AMD Phenoms in this regard. AMD uses Hypertransport, which is a point to point protocol so each core has dedicated bandwidth. Intel Core 2s (but not Core i7s and other Niehalem based CPUs) use a front side bus, which is shared bandwidth so you have more cores competing for the same bandwidth.

This can potentially make a same clock quad core (marginally) slower in some circumstances. Value for money is still with dual cores IMHO. That being said, I have a quad core.

I like how Donald Knuth sums it up:

To me, it looks more or less like the hardware designers have run out of ideas, and that they’re trying to pass the blame for the future demise of Moore’s Law to the software writers by giving us machines that work faster only on a few key benchmarks! I won’t be surprised at all if the whole multithreading idea turns out to be a flop, worse than the "Itanium" approach that was supposed to be so terrific—until it turned out that the wished-for compilers were basically impossible to write.

Let me put it this way: During the past 50 years, I’ve written well over a thousand programs, many of which have substantial size. I can’t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading. Surely, for example, multiple processors are no help to TeX.

For some applications, it's very easy to take advantage of multiple cores. But some other applications will never benefit from them, while the others might benefit if the developers optimize them (which is very difficult).

For my main desktop at my employer, I use a dual quad-core Xeon machine with 8 GB of RAM.

When I am programming, and I have Internet Explorer, Chrome, TweetDeck, Visual Studio 2008 (or 2010) and a local Sql Server Express instance... everything hums along just fine.

Comparatively, I had a dual core before, and things would start to crawl with just Visual Studio, Chrome, and Sql Express running.

Its a matter of what you do with the machine. If you are a power-user that will be video editing, 3-d modelling, or programming with significant resources.. then yes, you will want the quad-core and lots of RAM.