Completely disjoint iSCSI networks vs dedicated switches and VLANs
I'm looking for best practice or pros/cons for deploying an iSCSI storage network.
The two options are:
A completely disjoint iSCSI storage network
Dedicated switches with a separate VLAN for storage, but otherwise connected to production network.
Which of these approaches is generally recommended and why? If there is no general consensus of best practice relating to this, when is it appropriate to consider each option?
We use the same switches for regular ethernet traffic and iSCSI without any problems. You just gotta make sure that the regular ethernet traffic don't affect the iSCSI traffic in any way. Modern Cisco switches are usually line-speed (as in a 24 port 1gbit switch with a 24gbps backbone), so that you can in theory utilize 1gbps per port without anything getting hurt by it.
One word of advice if you're going to share the switches - make sure that you are 100% confident that the iSCSI traffic actually lands on the switch you want to. Our previous admin had a trunk between the two switches, and had the wrong nic's on the wrong switch. The end result was a major outage when that trunk went down.
My way of doing this is to limit the ports. We use access ports again the SAN with a specific VLAN, and on the trunkports to ESXi we use "switchport trunk allow vlan 123" etc. Same for the uplinks, where we set the trunks to NOT allow the iSCSI vlan's to be 100% sure of iSCSI traffic staying isolated.
Edit: Oh, and don't share switches if they are doing L3 - atleast not with access switches like 2960's. You really do not want to end up in the situation where the CPU goes through the roof and kills your ASIC.