For L2ARC and ZIL: is it better to have one large SSD for both, or two smaller SSDs?

Solution 1:

Short answer, since I don't see what problem you're looking to solve...

If you can, use separate devices. This depends on the scale of your environment... If it's just a simple home system or a virtualized or an all-in-one ZFS solution, you can use a single device.

In larger or high-performance ZFS solutions, I use devices suited specifically for their ZIL or L2ARC roles... E.g. STEC ZeusRAM or DDRDrive for ZIL and any enterprise SLC or MLC SAS SSD for L2ARC.

• ZIL devices should be low-capacity, low-latency devices capable of high IOPS. They are typically mirrored.
• L2ARC devices should be high-capacity (within reason: You need to add RAM as L2ARC size increases). They scale by striping.

What are you doing?

Solution 2:

There are some fundamental misconceptions from the outset about ZIL which need correcting before continuing.

Understand this: Under "normal" circumstances, ZIL/SLOG is not touched.

It's only written to when synchronous writes are commanded or if sync=always is enabled on a particular pool/dataset ("zfs get sync pool/dataset")

ZIL is never read from under normal circumstances. It's a disaster recovery feature.

IE: The ZIL is only there for when the power goes off. It's used to replay data which had been acked back to the OS before that data was committed to the pool. All ZFS writes to the pool (sync or async) are from memory buffers.

Under normal circumstances once the data hits the pool, the slog entry is allowed to evaporate - it's just a big circular write buffer and it doesn't need to be very large (even 1GB is overkill in most circumstances)

Non-synchronous writes are buffered in ram, collated and written to disk at an opportune moment. if the power goes off, that data is lost but the FS integrity is maintained (this is why you may want to set sync=always)

On the other hand, L2ARC is heavily hammered at both read and write level.

There's such a thing as "too much l2arc", because the metadata for what's in l2arc comes out of your ARC ram (ie, if you boost l2arc size you must boost ram to suit. Failure to do so can result in severe performance degradation and eventually l2arc usage will level off at some level well below "all the available space")

Despite the protestations of some manufacturers, you cannot make up a memory shortfall by boosting the l2arc sizes (Several makers of hardware raid arrays who've branched into ZFS appliances have made this assumption)

tl;dr: If your IO load is database activity then ZIL is likely to be slammed hard. If it's anything else then it is likely that it will only be lightly touched. It's highly likely that in 99.9% of activity ZIL functions never kick in.

Knowing that will allow you to decide if you need a SLOG partition for ZIL, whether it can cohabit with the l2arc partition or if it needs a standalone drive (and what performance level that standalone drive should be).