Recovering a nested PV

Solution 1:

The VM just got write I/O errors when the space in the thin pool was exhausted. For a VM this looks like hard disk unexpectedly denied all writes. So if the VM was bare hardware, the first action was to be find the new hard disk and clone this bad one into it. After the HW is fixed, you may fix logical structures.

In case of virtual machines, you don't have any broken hardware, you can "fix" the "hard disk" by restoring thin volume operation. Just enlarge the thin pool, use lvextend on the thin pool LV to add some space.

And, when it is done, boot the VM from some recovery (virtual) media and do standard file system recovery. Remember, there couldn't be much difficulty; modern filesystems generally designed to withstand this kind of failure.

Monitor the thin LVM. While data space exhaustion is not such a big problem, the metadata exhaustion might have much bigger impact. Don't allow this to happen.