LVM after vgcfgrestore got: device-mapper: reload ioctl on (254:19) failed: No data available

Server have Thin LVM with a few volumes:

vm-130-disk-0 - was deleted and need to be restored.
vm-137-disk-0 - was NOT deleted.

Trying to restore /etc/lvm/archive/pve_00336-2034680334.vg which created before deleting:

# vgcfgrestore --force --file /etc/lvm/archive/pve_00336-2034680334.vg pve
# vgimport pve
# lvchange -ay /dev/pve/vm-130-disk-0
      Thin pool pve-data-tpool (254:6) transaction_id is 324, while expected 311.
      ...

# lvs -a              
  LV              VG   Attr       LSize   Pool Origin Data%  Meta%
  data            pve  twi---tz--   1.57t      # NOT activated pool data
  [data_tdata]    pve  Twi-a-----   1.57t      # OK  a=Activated
  [data_tmeta]    pve  ewi-a-----  16.00g      # OK  a=Activated                                              
  root            pve  -wi-a-----  10.00g      # OK  a=Activated                                 
  vm-130-disk-0   pve  Vwi---tz--  32.00g data # NOT activated deleted volume
  vm-137-disk-0   pve  Vwi---tz--  22.00g data # NOT activated non-deleted volume
  ...

Well, here we failed because transaction missmatch between tmeta and tpool. Most answered people in Internet have mirror situation: tpool=312 and tmeta=324 and looks like transaction_id fix in .vg file helps them. Let's try to fix .vg file and activate:

Changed by hands transaction_id from 311 to 324 in /etc/lvm/archive/pve_00336-2034680334.vg ..

# vgcfgrestore --force --file /etc/lvm/archive/pve_00336-2034680334.vg pve
# vgimport pve
# lvchange -ay /dev/pve/vm-130-disk-0
   device-mapper: reload ioctl on (254:19) failed: No data available

In debug log appears: pve-vm--130--disk--0: Skipping NODE_DEL [trust_udev]

# lvs -a
  LV              VG   Attr       LSize   Pool Origin Data%  Meta%
  data           pve  twi-aotz--   1.57t             5.86   0.44  # OK
  [data_tdata]   pve  Twi-a-----   1.57t                          # OK                      
  [data_tmeta]   pve  ewi-a-----  16.00g                          # OK                      
  root           pve  -wi-a-----  10.00g                          # OK                      
  vm-130-disk-0  pve  Vwi---tz--  32.00g data                     # NOT activated deleted volume
  vm-137-disk-0  pve  Vwi-a-tz--  22.00g data        67.91        # OK activated non-deleted volume
  ...

"No data available" for deleted volume. Sad. AFAIK tpool has transaction_id=324 and I need somehow rollback tpool to 312. No idea how.

What can I to activate pve/vm-130-disk-0 ?

# lvm version
  LVM version:     2.02.168(2) (2016-11-30)
  Library version: 1.02.137 (2016-11-30)
  Driver version:  4.35.0

# uname -a
Linux adminslotlogicrestoreasap 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64 GNU/Linux

Thanks for reading! I will be grateful for any advice.

Solution 1:

Thin LVM archive files /etc/lvm/archive/*.vg have no physical extents in segments, but only device_ids. The mapping between device_ids and physical extents on block device stored in LVM metadata and can be dumped from inactive pool:

vgimport pve
lvchange --yes -ay pve/data_tmeta
thin_dump  /dev/mapper/pve-data_tmeta -o thin_dump_pve-data_tmeta.xml
lvchange       -an pve/data_tmeta

Thanks to lvremove you can see no deleted device_ids.

So Thin recovery is not possible in described case.

See also developer feedback (2014).