incomplete recursive snapshots on zfs
I got a bunch of new disks on one of our systems and wanted to transfer an existing pool over to them so what I did was this:
zfs snapshot -r old-pool@replicaton
zfs send -R old-pool@replication | mbuffer -m 1G | zfs receive -F -d new-pool
but then halfway through the operation, I got a warning from zfs send
, complaining
that the snapshot old-pool/some/fileset@replication
would not exist ...
when I went to investigate, I found indeed that zfs snapshot -r
had
neglected to create a snapshot on old-pool/some/fileset
. So I
ran
zfs list -r -o name old-pool | \
xargs -n1 perl -e 'system "zfs","list",$ARGV[0]."\@replication"'
and found that there were about 10% of the filesets which were lacking this snapshot ...
I then proceeded to create the missing snapshot individually, and it worked fine.
I have since repeated the experiment and found the same problem again ...
Any idea how this can be?
This is happening on omnios r151010
OH MY! How embarrassing ... I found the reason ...
The pool (old-pool) I am trying to replicate is itself a backup store, receiving regular updates via zfs receive -F
So while my recursive snapshot was initially complete, it got "cleaned out" as new snapshots were being integrated into individual filesets.
Once I stopped the backup script that was sending snapshots to old-pool
, the world was working as expected again.