SAN with a few SSD drives, or lots of old fashioned HDDs?
I can speak on the specifics of what you're trying to accomplish. Honestly, I would not consider an entry-level HP P2000/MSA2000 for your purpose.
These devices have many limitations and from a SAN feature-set perspective, are nothing more than a box of disks. No tiering, no intelligent caching, a maximum of 16 disks in a Virtual Disk group, low IOPS capabilities, poor SSD support, (especially on the unit you selected).
You would need to step up to the HP MSA2040 to see any performance benefit or official support with SSDs. Plus, do you really want to use iSCSI?
DAS may be your best option if you can tolerate local storage. PCIe flash storage will come in under your budget, but capacity will need to be planned carefully.
Can you elaborate on the specifications of your actual servers? Make/model, etc.
If clustering is a must-have, another option is to do the HP MSA2040 unit, but use a SAS unit instead of iSCSI. This is less costly than the other models, allows you to connect 4-8 servers, offers low-latency and great throughput, and can still support SSDs. Even with the Fibre or iSCSI models, this unit would give you more flexibility than the one you linked.
The rule of thumb I use for disk IO is:
75 IOPs per spindle for SATA.
150 IOPs per spindle for FC/SAS
1500 IOPs per spindle for SSD.
As well as IOPs per array also consider IOPs per terabyte. It's not uncommon to end up with a very bad IOP per TB ratio if doing SATA + RAID6. This might not sound too much, but you will often end up with someone spotting 'free space' on an array, and want to use it. It's common for people to buy gigs and ignore iops, when really the opposite is true in most enterprise systems.
Then add in cost of write penalty for RAID:
- 2 for RAID1, RAID1+0
- 4 for RAID5 (or 4)
- 6 for RAID6.
Write penalty can be partially mitigated nice big write caches and in the right circumstances. If you've lots of sequential write IO (like DB logs) you can reduce those write penalties on RAID 5 and 6 quite significantly. If you can write a full stripe (e.g. one block per spindle) you don't have to read to compute parity.
Assume a 8+2 RAID 6 set. In normal operation for a single write IO you need to:
- Read the 'updated' block.
- Read the first parity block
- Read the second parity block
- Recompute parity.
- write all 3. (6 IOs).
With a cached full stripe write - e.g 8 consecutive 'chunks' the size of the RAID stripe you can calculate parity on the whole lot, without needing a read. So you only need 10 writes - one to each data, and two parity.
This makes your write penalty 1.2.
You also need to bear in mind that write IO is easy to cache - you don't need to get it on disk immediately. It operates under a soft time constraint - as long as on average your incoming writes don't exceed spindle speed, it'll all be able to run at 'cache speed'.
Read IO on the other hand, suffers a hard time constraint - you cannot complete a read until the data has been fetched. Read caching and cache loading algorithms become important at that point - predictable read patterns (e.g. sequential, as you'd get from backup) can be predicted and prefetched, but random read patterns can't.
For databases, I'd generally suggest you assume that:
most of your 'database' IO is random read. (e.g. bad for random access). If you can afford the overhead, RAID1+0 is good - because mirrored disks gives two sources of reads.
most of your 'log' IO is sequential write. (e.g. good for caching, and contrary to what many DBAs will suggest, you probably want to RAID50 rather than RAID10).
The ratio of the two is difficult is hard to say. Depends what the DB does.
Because random read IO is a worst case for caching, it's where SSD really does come into it's own - a lot of manufacturers don't bother caching SSD because it's about the same speed anyway. So especially for things like temp databases and indexes, SSD gives a good return on investment.
Your analysis is pretty correct.
Use a few HDDs for lots of GBs, and lots of HDDs to a few IOps.
Use a few SSDs for lots of IOPs, and lots of SSDs for a few GBs
Which is more important for you? Space is the big cost-driver for SSD solutions, since the price-per-GB is much higher. If you're talking about a 200GB database needed 4K IOPs, a pair of SSDs will get you there. Or a 24 disk array of 15K drives, leaving you lots of space for bulk storage.
How many IOps you'll actually get out of those SSDs varies a lot based on the storage infrastructure (ewwhite will elaborate on that), but it's reasonable to get those kinds of speeds. Especially with Raid10, where parity isn't being computed.