RAID 10 - many slow disks vs fewer fast disks

I am looking to buy HDDs for a single server which will be used as development server. The server will use ESXi as hypervisor and will have some but not too many virtual machines (web server, database server, file server)

We have already taken a Samsung 950 Pro 256 GB for the virtual machines but I struggle in finding an answer to the question:

In a RAID 10: Are lots of slow disks (5400/5900 rpm) faster than fewer fast disks (7200 rpm)? For example 8x 5400rpm vs 4x 7200rpm

Would the 8 slow be a better choice compared to the 4 fast?

My choice was between 1TB WD Red which run at 5400rpm vs 2TB WD Se 7200rpm.

This is hard to answer directly, because the answer is very much 'it depends on your caching'.

You see - hard disks are slow. Very slow. But actually, most of the time you don't notice, if you have a decent cache, and a reasonably predicable access pattern.

Read caching means recently accessed things will be coming off cache. Write caching means that recently written things can be deferred and coalesced into full stripe writes.

But when it comes to measuring speed - there's two elements to it. Write performance ... actually probably doesn't matter too much, because of the caching - you can always cache writes, and so as long as your sustained write IO doesn't exceed that of your combined RAID group, it's irrelevant.

But read IO it's much more of a problem - you cannot service a read until you've got your data, and to get your data you have to wait for the disk if it's not already in cache. This is where you see the performance cost - it shows up in read latency.

And that's why slower disks are a problem - because it doesn't actually matter how fast the combined RAID group is, when the data is at rest on one of two drives - you have to access that drive to service the read.

So - at risk of sounding a bit wishy washy - "it depends" a lot on your workload. You can usually get by with some really slow disks and some awesome caching if you want really large amounts of terabytes cheap.

However, I'd only bother doing so when we start talking at 100+TB or so, which is where the cost of rack space, cooling, floor footprint etc. start to become significant.

For your application - I'd say buy the fastest drives you can afford, because it's a lot easier to buy more drives later if you need them, than to realise you can't because you've filled your drive bays and need a new shelf.

But I'd strongly suggest also considering SSDs - their price-per-gig isn't amazing, but their price-per-IOP really is. And whilst no one seems to care about IOPs, that's the thing they really do care about when it comes to 'being a bit slow'.

Based on calculations from RAID Performance Calculator, eight 5400rpm disks will yield about 304 IO/s. Whereas four 7200rpm disks will yield about 210 IO/s.

The main difference between your two choices is 3 year warranty vs 5 years and SOHO usage area vs. certification for datacenters. In terms of raw I/O performance, there is not much of a difference between 5400/5900 and 7200 rpm S-ATA disks nowadays (that was different a few years ago).

In the end therefore the array with more disks will very likely show a superior performance, as RAID10 scales with the number of disks.

What I would guess is more important in this setup, then the number of drives is, get a proper hardware RAID controller with BBU+Cache if you can afford it - or even better go with SSD only from the start on.

Also please be aware, that ESXi natively only supports hardware RAID! While there are workarounds to use software RAID with ESXi, I personally wouldn't advice using these, if you are looking for a decent performance.

Protip: If you are on a really tight budget to self-build this storage server and we are talking about a non mission-critical system, then go fetch yourself an OEM version of a proper RAID controller (IBM M5015, DELL PERC H710, etc.) on ebay.