When using thin-provisioning with ZFS, how do you make sure you don't run out of physical disk space?

On the Nexenta side, there's a volume-check script that's setup to run hourly by default. It will:
Check volume health and capacity, clear correctable device errors, validate mountpoints.
It also sends a weekly summary report via email.

However, there are some things you should consider when planning a Nexenta storage solution for the purposes you've listed.

• You may want to consider having multiple pools for flexibility. A single pool works, but sometimes it's necessary to move data around or just have the option of a second pool on local storage.
• ZFS zvols can be expanded/contracted on the fly. For instance, if you allocate 20TB to a thin-provisioned zvol, you can change it to 30TB or 100TB very easily. You don't need to over-provision 100TB for the future if you don't have it at present.
• With thin-provisioned zvols, once the space is used, you can't reclaim it. If you thin provision a 2TB zvol in a 10TB pool, fill the zvol up, then delete the VMs on that zvol, your pool will still only show 8TB free. That 2TB is going to remain.
• Will you be using ZFS compression or deduplication or both? One situation where it DOES make sense to over-provision is if you're using inline compression and highly-compressible data. Same for data that is deduped. In my case, the data sets I work with compress 60%-80%, so I present larger zvols than the amount of storage I actually have.
• Using mirrors versus raidz1/2/3 makes it easier to expand underlying storage. You can add mirrored disk pairs to a zpool, but you can't expand raidz1/2/3 unless you add another vdev (group of raidz(x) disks). You'd also want to rebalance the data within to redistribute across the disks.
• Which virtualization technology will you be using? If VMWare, you can thin-provision. You will see datastore warnings near 80% utilization, I believe. VMware also complains if you're in a dangerous situation with snapshot size growth.
• If you are doing a lot of VM testing, or have VMs that fluctuate in size, I'd suggest using iSCSI and zvols for the relatively static VMs and NFS for the test VMs (if that's an option for your preferred virtualization solution). With NFS, you can make more efficient use of your storage space since you see the zpool's full available size and don't have any size ceiling to worry about.

In short... I wouldn't overprovision to account for future growth. It's not necessary. There are hourly checks in Nexenta to alert to space utilization. Also think about whether you will use compression or not (deduplication requires a bit more planning). test things out and see what the VM footprint will look like before going into production. It will be more difficult to change afterwards.

If you have some monitoring system like Nagios in place, you easily could write a check evaluating the output of zpool list and checking it against thresholds within your comfort zone.

If you don't have a monitoring system, you should use this opportunity to install one - a SAN is a critical piece of infrastructure equipment which needs constant monitoring if you don't want end up with downtimes or data loss due to defective disks, out-of-space conditions, hardware failures or connectivity problems.