ZFS online disk migration

I'm wondering if ZFS can handle the following. Say I have FreeBSD installed on a zpool with two physical 500GB disks in a mirrored setup. These form the root partition, so only a maximum of 20GB is actually being used.

I would later want to replace the physical disks with two small SSDs in a mirrored setup. Is there a way to do this online? My idea was to add these disks to the zpool as extra mirrors, wait for resilvering to finish and then remove the old drives. Problem is that these new disks would be smaller than the originals.

An idea that I had is to just partition the 500GB drives, so that ZFS only gets, say, 30GB of space on each physical disk the rest being wasted. This way the SSDs would have more space, so I would not arrive at a shrinkage problem. Problem is that this way I need to estimate in advance the size of the future replacement disks. Is there a way to avoid this?

Solution 1:

You are correct that if the vdev is the whole disk, when you switch to SSDs they would have to be at least that size, you can't "shrink" to smaller disks (of course you can grow to larger disks). You are also correct that you can use partitions so the vdevs are smaller than the future SSDs, and you'd have to guess at how large those SSDs are going to be.

Alternatively, if the server can accommodate all 4 disks at the same time, even for a short period of time, you can create the zpool any way you want now, then use send/receive to transfer the data to the new SSDs. This does require that all the devices be plugged in at the same time. A small caveat, the send/receive operation can only send from a snapshot - so you can't duplicate a "live" disk and will need to make accommodations for this (the downtime can be quite short by adding a little complexity).

Even more complicated it's actually possible to store the "send" to an intermediate file on another disk somewhere, then "receive" from that. So you only need 3 disks in the server at any time. And more complicated yet, the 4 disks could be on different servers and send/receive can be piped through ssh or even netcat.

Solution 2:

I can't think of any obvious way of doing this.

For now, seeing as you have to partition the root disks anyway (to add bootcode), and it doesn't sound like something that should happen often, I would suggest just creating a small(<20GB) partition for the root pool. If you need extra space you can create a second pool out of the space left over. When it comes to put the SSDs in, just partition them, add bootcode, 'zfs attach' them to the root pool, then 'zfs detach' the originals (or 'zfs replace' instead of 'zfs attach' and 'zfs detach').