What is typical SAN performance?

I have very little experience with SANs, so please excuse this newbie.

One of our production sites has an HP EVA8000 SAN with a bunch of disks (forget exactly how much). it has been configured into a bunch of raid5's and the SQL server uses one RAID volume for data and another for the Log.

So far so good. Or so I thought

Today i ran some speed tests. Using rdfc i created a 100 MB file on the main data disk while the system was heavily used. This showed between 5 and 20 MB/s throughput for write, which, i thought was a bit on the low side. I then took down the application for maintenance, and ran the test again. This time on a quiet system I got 50MB/s write performance.

Is that not a bit on the low side? I mean, I get the same from the SATA drive in my desktop.

I then tried writing 500MB to one drive (the data drive), while writing 500MB to another drive (the log drive). Those two writes interferred with each other. Is that a sign of something wrongly setup?

I don't know what I had expected, but I'm currently reviewing the system to try and locate bottlenecks.

So, i guess my question would be. What kind of performance would you expect (read/write) from a (probably) expensive SAN using Fiber connectors?

(No, its not the little netgear thingy; its this) http://h10010.www1.hp.com/wwpc/ca/en/sm/WF05a/12135568-12135820-12135820-12208992-12208992-12236532.html

Solution 1:

We have an EVA6100 so I've done tests on similar hardware. The 8000 is, IIRC, an older system than the 6100.

First off, your writes to the DB volume and Log volume are very likely contending for writes as it sounds like they're on the same Disk Group inside the EVA. A disk group is a grouping of disks that you create LUNs on, and each LUN is a disk presented to one (or more) servers. It is not at all uncommon to house many, many LUNs in a single disk-group. If you only have one disk-group, then the benefits of separating your log I/O from your DB I/O are somewhat reduced.

Second, if that DG is running into true I/O contention, then it'll degrade performance. This is something that's very hard to check from a consumer point of view, it's the kind of thing the SAN maintainers need to be aware of. Ask them if they know what the average controller CPU load is, as that'll significantly affect your write performance.

Third, know what your I/O mix really is an benchmark for that, not just file-copy speed. If your DB volume I/O is 80/20 read/write, then you really want to check read-path throughput. If it's the other way around, then your write performance is the metric you really need to check.

Solution 2:

By simply writing a large file you do not measure storage performance. You just measure the performance of writing a large file. You are sadly mistaken if you think you will get anything similar when: writing randomly all over the disk, writing many blocks simultaneously, writing really large or really small blocks. I guarantee you, that you can have from your array max 1 MB/s on one type of workload, and then with another workload/setup you will easily get 800 MB/s (...provided your server can handle that much).

If you want to learn about these, use a proper tool. I've used Intel IOMeter at the time and I learned a lot with it. Probably now there are some better tools. The main parameter to watch out for is the queue size and random-vs-sequential operation. Read about disk mechanics, why the seek time is so unbelievably slow, and how RAID5 functions.

If you want to know how your storage is performing in real life, take a look at PhysicalDisk counters in perfmon.exe during your work day. Then create some load on your actual application and observe what happens.