ZFS SAS/SATA controller recommendations
I've been working with OpenSolaris and ZFS for 6 months, primarily on a Sun Fire x4540 and standard Dell and HP hardware. One downside to standard Perc and HP Smart Array controllers is that they do not have a true "passthrough" JBOD mode to present individual disks to ZFS.
One can configure multiple RAID 0 arrays and get them working in ZFS, but it impacts hotswap capabilities (thus requiring a reboot upon disk failure/replacement). I'm curious as to what SAS/SATA controllers are recommended for home-brewed ZFS storage solutions. In addition, what effect does battery-backed write cache (BBWC) have in ZFS storage?
Solution 1:
I recommend a (almost) any SAS/RAID controller that uses the LSI 1068E SAS chip (PCI-E) and supports Initiator-Target mode (JBOD mode). This does not include MegaRAID controllers.
I use the a 3442E and Dell 6/iR.
- LSI SAS 3442E (4 internal - SFF-8484, 4 external - SFF-8470)
- LSI SAS 3081E (8 internal - SFF-8087)
- LSI SAS 3041E (4 internal - SATA)
- LSI SAS 3801E (8 external )
- LSI SAS 9212-4i4e (4 internal - SATA, 4 external SFF-8088)
- LSI SAS 9211-8i (8 internal - SFF-8087)
You get the idea.
Also I recommend Dell SAS Controllers which are cheap on ebay.
- Dell SAS 5/iR - (4 internal ports)
- Dell SAS 6/iR - (8 internal ports)
- Dell H200
Not - Dell PERC 5/E - Dell PERC 5/I - Dell PERC 6/E - Dell PERC 6/I
Why not MegaRAID?
These controllers do not support a non-RAID, JBOD mode. They cannot present disks to the host directly. Each disk must first be included in a Logical disk. You can create a single logical volume for each disk you have and give this to ZFS. Performance should be equivalent. However, it means putting header information onto the physical disk. If your controller fails, you must use another MegaRAID, configured exactly the same. And there is not guarantee that another SAS or SATA controller will see the data correctly and present the volumes in a way that the zpool can be correctly constructed.
Solution 2:
The recommened controller is anything using the LSI SAS 1068E chipset. It's what Sun uses, so has the best support. As phresus mentions, the ASOC-USAS-L8i is a good cheap card that uses it. Battery-backed write cache helps performance, particularly for NFS. Another alternative is to use an SSD for the ZFS Intent Log, but only some have capacitors to ensure data is written after power-loss. For large read workloads, a MLC SSD can be used as L2ARC to supplement RAM caching.
Solution 3:
I would highly recommend you check out the Guide to Building a Media Storage Server thread at AVS Forum here. There is a wealth of knowledge in there comparing/contrasting various storage methods including ZFS. From the thread it seems like the Supermicro aoc-sat2-mv8 or newer aoc-saslp-mv8 are currently the most economical solutions for JBOD.