How smart are RAID controllers/cabinets?

How smart are RAID controllers when they're under load?

Given a moderate-to-high-end controller and cabinet (i.e. current standard Dell or HP off-the-shelf kit...), RAID 5, 10gb fiber, at least 3/4 utilized space, and lots of small non-sequential with heavy mixed read/write access to files such as would be found on a file or email server...

Practical implementation question: Given the same amount of space, what would be faster, a small number of large, high speed drives (i.e. 4x 1TB 15,000 rpm) or a larger number of smaller, moderate or low speed drives (i.e. 9x 500GB 7200 or 10,000 rpm) drives?

Theoretical question: Do RAID cabinets/controllers know where the head of a drive currently is and the location they need to seek to so that they can assign reads to the drive with the least head-travel distance? Or does it matter?

What other variables come into play with minimizing response time and maximizing throughput with a large numbers of non-sequential small files on a shared storage array? Note that cache doesn't come into play that heavily because of the nature of the data.

Solution 1:

From your description your workload will be heavily random access, so the limiting factor is random IO operations per second. On a RAID-5 you will get (in practice) slightly less than one I/O read or write per revolution of the disk per physical spindle. In this situation more physical disks and faster RPM means more throughput.

Under a heavily random I/O situation where the working set of the data requests overflows the cache the throughput of the system is a function of the number of physical disks x the speed of the disk. The more disks the better, the faster the better.

On your theoretical question, the disks support a feature known as 'tagged command queuing'. This allows the controller to dispatch I/O requests to the disks and recieve the requests asynchronously. The internal board on the disk is aware of the position of the disk heads and can optimise the operations by completing them in the order it decides is optimal. The algorithm for this is some variant on 'Elevator Seeking'

Results can be returned out of order, but are tagged with the request number so the RAID controller knows which request to match the reply to (hence the 'tagged'). SATA has a slightly different protocol known a 'native command queuing' that does something similar.

In this case the RAID controller does not have to be aware of the physical position of the disk heads as this is managed by the firmware on the disk itself.

On a heavily random access workload a single pair of FC loops will support quite a lot of disks. For a streaming workload such as video FC will become a bottleneck more quickly.

Some controllers can support impressively large cache sizes. You may want to try and estimate the working set size and see if you can actually upgrade your controller's cache to accomodate it. If you have a background in statistics you might be able to build a monte-carlo model based on usage statistics gathered from the requests.

Another possiblility to improve performance might be to use a layer of solid-state disks for fast storage, although this depends on whether your controller supports this configuration.

Solution 2:

IOPS, IOPS and IOPS. Those are what you need to consider. You might get more IOPS from a faster spinning drive. Alternatively you might get more IOPS from a higher spindle count.

There is a good comparative article by Adaptec that answers almost exactly the same question.

If you have some drives in mind (ie. from a specific vendor) then you can run the maths.