iSCSI SAN - Network adapter bottleneck
My new database server (Windows Server 2008 R2) has a 4 Gbps connection to my Dell EqualLogic iSCSI SAN. I've been testing it by backing up multiple databases at the same time. There appears to be a bottleneck with the network adapter. It shows up as 4 1 Gbps connections, and the network utilization never exceeds 2 Gbps. Any recommendations where to look next?
SAN: 48 drives. 10K 600GB SAS. Raid 50.
Update:
Flow control is set to auto, but I think it is being used because MPIO is installed. There is a switch and I'm working on jumbo frames, but while researching MPIO I noticed something that might be affecting performance.
In the database backup test I was performing, 4 volumes were involved: one for data, one for the log, and 2 for the backup files (split by RedGate SQL Backup to improve performance). I was trying to determine if there was any performance benefit to separate volumes on a SAN. On my old server, I would use 4 different RAID arrays in this situation.
Each of my 1 Gbps network adapters has an IP assigned in the range 10.0.1.251-10.0.1.254. Each volume has 2 of these IP addresses assigned to it in the "Dell EqualLogic MPIO" tab in iSCSI Initiator Properties. 251 & 252 are used for the log & both backup volumes. 252 & 253 are used for the data volume. It seems like this could be a limiting factor.
I'm going to see if I can assign them different IP addresses.
Update 2:
Using the MPIO Remote Setup Wizard, I changed the "Max sessions per volume slice" from the default 2 to 4 and rebooted. Each volume now has 4 paths, one for each NIC. The utilization is now fairly evenly distributed between the 4 NICs, but the total is still less than 2 Gbps. Each one maxes out at about 45%. Once I get access to my switch I will try to get jumbo frames working and see if that helps.
Solution:
Jumbo Frames. They had to be set on the database server NICs and on the switch. The SAN handled it automatically. This required a reboot on the DB server.
What kind & how many hard disks does your SAN have? 2Gbps is around 250MB/sec which is very decent performance if you have a pretty large array of 7200rpm SATA drives.
If you need higher performance you should look into equipping your SAN with SSD disks which offer higher I/O.