iSCSI for databases, good idea

Solution 1:

Your question needs to be broken down into two parts: internal vs external storage, and iSCSI vs the alternatives (FC, FCoE, NFS). My experience is with large corporate clients running mainly Oracle, so this may not apply to smaller environments or other databases. In my opinion, there is a lot of value in external storage; iSCSI is one option for providing this but it is not as mature as some of the alternatives.

Internal vs External Storage

External storage, generally provisioned in a disk array and presented through a storage area network (SAN), provides a number of advantages:

• Fast performance, often with hardware-accelerated RAID and large battery-backed caches.
• Easy to scale up volumes to use more disks for performance and capacity.
• Centralized access to storage resources to avoid silos of wasted resources.
• Able to share storage for high-performance or high-availability clusters (e.g. Oracle RAC).
• May come with features for snapshotting and replicating data to remote sites.
• May come with decent analytics tools to track performance.

The main disadvantages of external storage are the complexity and cost of setting up and maintaining the storage area network and the storage arrays.

iSCSI vs Alternatives

Fibre Channel is currently the standard mechanism for accessing external storage for databases. It is common to see SAS (Serial-Attached-SCSI) used for less critical data, more as an extension to internal disk than as disk in a SAN. The key point about these technologies is that they run on dedicated storage networks.

Newer alternatives such as FCoE and iSCSI provide effectively the same protocol as FC and SAS, except they run over Ethernet and therefore can use the same infrastructure used for host-to-host networking. The idea is that by converging on Ethernet, companies can reduce the cost and complexity of having external storage. However, there are still questions around whether Ethernet as a transport provides the speed and reliability of the dedicated technologies.

NFS is a file-level protocol which also runs over Ethernet. In the past it has been considered to have too much overhead for databases, but with hardware offloading in newer network adapters, better OS network stacks, and direct support by the database (e.g. the Direct NFS feature in Oracle) it is also a viable option for some companies. NFS is particularly nice because it reduces the administration involved in resizing volumes, and also fits the model of virtualized storage such as EMC VNX and Oracle ZFS Storage.

Solution 2:

In my opinion iSCSI is less stable of a storage network transport than Fibre Channel or Fibre Channel over Ethernet. Presuming you've made the needed optimizations, it should work. It'll work best if the iSCSI connection is going over a dedicated ethernet, no non-iSCSI traffic on it to get in the way. ISCSI initiators have gotten better over the years and even support multipathing now, if you want to go there.

As for running a transactional database over iSCSI, I have my doubts. A cold-standby server should be able to pick up where the hot server left off, presuming it left the database files in a recoverable state. That said, the one use-case where the dedicated storage transport networks really shine is in this sort of "lots of little transactions" use-case.

In the end, by my assessment, it'll work, but may fail catastrophically more often than using FC or FCoE.