Can I make snapshots of an APFS non-bootable volume?

Solution 1:

TL;DR: Use the hypervisor's native snapshots, and never use APFS snapshots on a volume with VMs.

Carbon Copy Cloner is a good utility to manually manage APFS snapshots. Open the program, switch the UI from Simple to Normal mode (in the menu). In the volume table in the lower left of the main window, you can click on any volume and a pane opens that shows the snapshots, and lets you delete old ones. You can create snapshots by running a backup task - even a dummy one that copies just one file to some folder.

The problem is that when you have a disk with VMs on it, you should only ever use one snapshot mechanism: either APFS snapshots, or the hypervisor's snapshots, not both. I suggest that you never use APFS snapshots on any disk with VMs, and use the hypervisor's native snapshots. They are a much more mature technology.

The snapshots work by retaining all of the old data, so their effective size grows over time - no data after the most recent snapshot's timestamp is ever removed from the disk. If you use hypervisor snapshots on top of APFS snapshots, the space usage doubles and you will run out of space while doing things that should conserve space.

For example, start with a single 600Gb VM, with VMware snapshots, on a 1TB APFS volume. You want to delete a 200Gb snapshot from that VM: VMware will integrate the snapshot into the higher level disk image. The problem is that this will effectively write 200Gb of "new" data to disk - APFS doesn't have deduplication!! Thus, the APFS snapshot will have to preserve the previous state of those 200Gb's worth. You think you should reclaim 200Gb of space, but you end up using 400Gb more, and get a full volume.

At least on macOS 10.13, AFPS has bugs and APFS volumes that run out of space due to APFS snapshot activity become unusable. You won't be able to delete the APFS snapshots, nor do anything else on that disk. It's nominally read-only, all write attempts will be failing with a weird "permission denied" error. Your only recourse will be to make a backup of the disk, then erase/reformat it (How do I know? It just happened to me and it's totally reproducible).