Will adding a SSD cache device to my ZFS storage improve performance?
The server has 4GB of RAM and my zpool is made up of 15.5k SAS drives arranged like this:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c0t2d0 ONLINE 0 0 0
c0t3d0 ONLINE 0 0 0
c0t4d0 ONLINE 0 0 0
c0t5d0 ONLINE 0 0 0
c0t6d0 ONLINE 0 0 0
c0t7d0 ONLINE 0 0 0
c0t8d0 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
c0t10d0 ONLINE 0 0 0
c0t11d0 ONLINE 0 0 0
c0t12d0 ONLINE 0 0 0
c0t13d0 ONLINE 0 0 0
c0t14d0 ONLINE 0 0 0
spares
c0t9d0 AVAIL
c0t1d0 AVAIL
The primary use is as an NFS store for a couple VMWare ESXi servers. I can't do any "true" benchmarks because this is a production system (no budget for test systems), but using dd
and bonnie++
I can't get more than ~40-50MB/s writes and ~70-90MB/s reads. It seems I should be able to do much better, but I'm not sure where to optimize. Based on what I've read, I think dropping in a OCZ Vertex 2 Pro SSD as my L2ARC is going to be the best bang-for-the-buck to improve througput.
- Is there something else I should be looking into to help performance? If not...
- How do I know how big a cache device I need?
- Am I safe with only a single SSD as my cache device?
My guess is that the ZFS Intent Log (ZIL) is limiting the write performance. A fast SSD used as a log device could help. Some hints:
- ZIL Accelerator: DRAM or Flash? explains the issues in detail.
- You may want to test with a disabled ZIL. See my other posting.
- Checkout ZFS Features in Oracle Solaris Express if you're using a recent ZFS release because the way how to disable the ZIL has changed. There's a new sync filesystem property now.
- Until recently it was impossible to remove a log device from a ZFS pool. Of course, this caused problems if the single log devices failed. It was therefore recommended to use a mirror for the log device. However, recent ZFS releases fixed this limitation and it's now possible to remove the single log device with
zpool remove
. Make sure you use a version that supports this feature if you really going to install the SSD.