incomplete recursive snapshots on zfs

I got a bunch of new disks on one of our systems and wanted to transfer an existing pool over to them so what I did was this:

zfs snapshot -r old-pool@replicaton
zfs send -R old-pool@replication | mbuffer -m 1G  | zfs receive -F -d  new-pool

but then halfway through the operation, I got a warning from zfs send, complaining that the snapshot old-pool/some/fileset@replication would not exist ...

when I went to investigate, I found indeed that zfs snapshot -r had neglected to create a snapshot on old-pool/some/fileset. So I ran

zfs list -r -o name old-pool | \
xargs -n1 perl -e 'system "zfs","list",$ARGV[0]."\@replication"'

and found that there were about 10% of the filesets which were lacking this snapshot ...

I then proceeded to create the missing snapshot individually, and it worked fine.

I have since repeated the experiment and found the same problem again ...

Any idea how this can be?

This is happening on omnios r151010


OH MY! How embarrassing ... I found the reason ...

The pool (old-pool) I am trying to replicate is itself a backup store, receiving regular updates via zfs receive -F

So while my recursive snapshot was initially complete, it got "cleaned out" as new snapshots were being integrated into individual filesets.

Once I stopped the backup script that was sending snapshots to old-pool, the world was working as expected again.