RAID Array performance on an HP Proliant ML350 G5 Smart Array E200i

We have a client who is complaining about performance of an application which utilizes an MS SQL database. They do not believe the performance issues are the fault of the application itself.

The Smart Array E200i RAID controller has 128MB cache and we have the cache set to 75% read/25% write. The disk array set to enable write caching.

Recently we ran a disk performance test using SQLIO based on this guide. We used a 10 GB file for the test found that the average sequential read rate was ~60 MB/sec (megabytes/sec) and the average random read rate was ~30 MB/sec. Are these numbers on par for what the server should be performing? Better than on par? Horrible? Amazing?

Additional information on the server set up/RAID controller config:
There are three, 146 GB SAS 10k RPM 3.0 GB/sec (model HP DG146BABCF) drives, configured in a RAID 5 array. These are the only physical disks available to the server so both logs and data, including operating system data and paging file are all on the same physical disk array (there are 2 logical drives with the OS data being separate). The array stripe size is set to 64k. Total usable space is 273 GB.

The HP Advanced Data Guard is turned off. Rebuild and expand priority are set to medium. Surface scan delay is 15 sec. The controller has a cache board and a battery pack.

Too many imponderables. For example, how are the disks set up? If the logs and data share the same disks the random I/O from the data areas will disrupt the log traffic, which is mostly sequential I/O and is disproportionately affected by a busy random access workload on the same disks.

Without some more insight into your configuration I can't really say what might be causing the problem.

For example, 60MB/sec off a RAID is about right for a 4-disk RAID-5 or RAID-10 with 64k stripes and 15k drives. Each drive will read one 64k stripe per revolution of the disk (about 250/sec for a 15k drive) which gives you 15MB/sec per drive.

The average seek time for a 15k disk is around 3ms across the whole disk. On a mostly contiguous 10GB file on a RAID volume with (say) 146GB or 300GB disks and a bit of help from the cache I could see 30MB/sec being a reasonable figure for a disk array configured as described above. It would indicate averaging a data read about every two revolutions of the disks.

That's a thought off the top of my head for a configuration one might reasonably expect to see on an ML350. However, I have no idea if that matches your actual configuration, so I can't really comment on whether the observations are relevant in your case.

The E200i has notoriously poor performance, as documented by Lukas here and Ryan here. Check you've got the optional battery kit (BBWC) attached and the HP Array Configuration Utility is showing the battery status is ok. (In HP Array Configuration Utility click on the Controller in the Configuration View in the center and choose More Information on the Common Tasks menu on the right). Having this Battery in place gives the disk controller a good boost in performance.

But your best bet is going to be to swap out the E200 with a 'proper' HP SmartArray controller, the P400 or P600's with the 512MB BBWC should give you a good speed boost.

Below is a benchmark we did with similar hardware, and a few differences noted. I would guess some of the performance hit is due to the partitions are misaligned (Windows 2003 misaligns partitions by default). Run the following command, and if the starting offset is 32256, it is misaligned.

wmic partition get index, blocksize, name, startingoffset

To properly align the partitions, you need to use the DISKPART utility.

Test Hardware:
HP DL380 G5
2 sockets, 4 total cores
16GB of RAM
HP P400 controller
512MB cache
25% Read / 75% Write
10K RPM HDDs
Windows Server 2003, 32-bit
Five-minute test cycle

Results (MBytes/second):
(RS: Read Sequential RR: Random Read WS: Write Sequential WR: Write Random)

Raid5 3 HD:
RS 180
RR 180
WS 120
WR 130

Raid5 4 HD:
RS: 240
RR: 260
WS: 175
WR: 180

Raid5 5 HD:
RS: 310
RR: 320
WS: 210
WR: 225

You should not be having a "pseudo-degraded" Raid5 as a database backing store. AFAIK is the slowest possible disk configuration. Add in a low end controller and things will not get better.

Hitachi AMS owners can ignore this posting (4disk raid5 = above 300MB/s). For the rest it probably applies.

Does the Smart Array E200i have a backup battery write cache installed? I have seen some dire performance out of Smart Array controlers without a BBWC.

As for the data rate readings you took, I can't really comment I have seen 60 MB/sec out of a high end desktop system. However data rate readings can lie! Post up the parameters for the tool you used for a better comparison.