What is better for a Java Web Application: more CPU cores or a higher clock speed?

tldr; The real answer is probably "more RAM", but as you've asked your question the answer is, of course, it depends. Then again, 32 cores @2.5Ghz will almost certainly beat 8 cores @3.8Ghz - it's 4 times more cores vs. 1.5 times faster clock. Not a very fair fight.

A few factors you should consider are transaction response time, concurrent users and application architecture.

Transaction response time If your Java application responds to most requests in a few milliseconds then having more cores to handle more concurrent requests is probably the way to go. But if your application mostly handles longer running, more complex transactions it might benefit from faster cores. (or it might not - see below)

Concurrent users and requests If your Java application receives a large number of concurrent requests then more cores will probably help. If you don't have that many concurrent requests then you might just be paying for a bunch of extra idle cores.

Application architecture Those long running requests I mentioned won't benefit much from faster cores if the app server spends most of the transaction time waiting for responses from web services, databases, kafaka/mq/etc. I've seen plenty of applications with 20-30 second transactions that only spend a small portion of their response time processing in the application itself, and the rest of the time waiting for responses from databases and web services.

You also have to make sure the different parts of your application fit together well. It doesn't do you much good to have 32 or 64 threads each handling a request all queuing up waiting for one of 10 connections in JDBC pool, aka the pig in a python problem. A bit of planning and design now will save you a lot of performance troubleshooting later.

One last thing - what CPUs could you possibly be comparing? The cheapest 32 core 2.5 GHz CPU I can find costs at least 3 or 4 times more than any 8 core 3.8 Ghz CPU.

Assuming your Java web server is appropriately configured, you should go for more cores.

There are still dependencies, like semaphores, concurrent accesses that will still have some threads waiting, whatever the number of cores or speed. But it's better when it's managed by the CPU (cores) than by the OS (multi-threading).

And anyway, 32 cores @2.5Ghz will handle more threads and better than 8 cores @3.8Ghz.

Also, the heat produced by the CPU depends on the frequency (among other things) and this is not linear. Meaning, 3.8Ghz will generate more heat than 3.8/2.5 x (has to be confirmed based on your exact CPUs types/brands... many sites offer detailed information).

You tell us that the a request takes about 100-200 ms to execute, and that it's mostly processing time (though it's difficult to separate what is actual CPU execution from what is in reality memory access), very little I/O, waits for databases, etc.

You would have to benchmark how long it actually takes on each of the two CPUs, but let's suppose it takes 150 ms on the slower CPU (with 32 cores) and 100 ms on the faster one (with only 8 cores).

Then the first CPU would be able to handle up to 32/0.15 = 213 requests per second.

The second CPU would be able to handle up to 8/0.1 = 80 requests per second.

So the big question is: how many requests per second do you expect? If you are nowhere near dozens of requests per second, then you don't need the first CPU, and the second one will give you faster execution time on each request. If you do need over 100 requests per second, then the first one makes sense (or it probably makes even more sense to have more than one server).

Note that this is very very back-of-the-envelope-type estimations. The only way to know for sure is to benchmark each of the servers with a real-life load. As stated above, fast CPUs or CPUs with lots of cores can quickly become starved for memory access. The size of the various CPU caches is very important here, as well as the "working set" of each request. And that's considering truly CPU-bound work, with no system calls, no shared resources, no I/O...