cannot remove faulty device from LVM2 raid1 mirror

One of the three disks went off in my LVM raid1 logical volume. (It was a bad experiment actually). The volume is a pure LVM raid1 one (no, not an LVM mirror), without any mdadm / fakeraid.

The problem was realized only at reboot, the device was completely missing. As I still have two healthy disks, I'd like to use the volume as a two-way raid1 for a while, and to replace the bad disk later. However, I cannot remove the missing device from the volume.

Here is the basic info about the topology. The volume group is vgQ3, while the logical volume is lvRAID1:

    #lvm pvs | grep vgQ3
      Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0.
      /dev/sdb2      vgQ3   lvm2 a--   79.31g 17.91g  
      /dev/sdc2      vgQ3   lvm2 a--   79.31g 17.91g  
      unknown device vgQ3   lvm2 a-m   79.31g 17.91g

    # lvm lvs -a -o +devices | grep vgQ3
      Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0.
      lvRAID1            vgQ3   rwi---r-p  61.39g   lvRAID1_rimage_0(0),lvRAID1_rimage_1(0),lvRAID1_rimage_2(0)
      [lvRAID1_rimage_0] vgQ3   Iwi---r--  61.39g   /dev/sdc2(1)                                               
      [lvRAID1_rimage_1] vgQ3   Iwi---r--  61.39g   /dev/sdb2(1)                                               
      [lvRAID1_rimage_2] vgQ3   Iwi---r-p  61.39g   unknown device(1)                                          
      [lvRAID1_rmeta_0]  vgQ3   ewi---r--  4.00m    /dev/sdc2(0)                                               
      [lvRAID1_rmeta_1]  vgQ3   ewi---r--  4.00m    /dev/sdb2(0)                                               
      [lvRAID1_rmeta_2]  vgQ3   ewi---r-p  4.00m    unknown device(0)

First I tried to reduce the volume group, but without success:

    # vgreduce --removemissing vgQ3
      Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0.
      WARNING: Partial LV lvRAID1 needs to be repaired or removed. 
      WARNING: Partial LV lvRAID1_rmeta_2 needs to be repaired or removed. 
      WARNING: Partial LV lvRAID1_rimage_2 needs to be repaired or removed. 
      There are still partial LVs in VG vgQ3.
      To remove them unconditionally use: vgreduce --removemissing --force.

    # vgreduce --removemissing --force vgQ3
      Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0.
      Persistent log is not supported on segment-by-segment mirroring

And so on... Then I tried to reduce the three-way raid1 device to a two-way one (Trying to reduce it to a one-way linear volume resulted in the same error message):

    # lvconvert -m1 vgQ3/lvRAID1
      Couldn't find device with uuid vG3BbG-ap9H-iYGg-qfE2-R13Z-rLJI-AJcAv0.
      Cannot change VG vgQ3 while PVs are missing.
      Consider vgreduce --removemissing.

Well, I've run out of the ideas I could try, except that destroying the volume and rebuilding it, which considerably would be more painful than simply reducing it to a two-way raid1.

(To make the story complete, by executing:

    # vgchange -a y -P vgQ3

I could create and mount the raid1 volume (/dev/vgQ3/..) in partial mode, so I could save the content. The next reboot makes the device disappear however, so the story continues again and again).

Is there any solution to such problems?

I had the exactly same error. I solved this by updating lvm version to lvm2-2.02.111-2.el6_6.1. Once you update the lvm, you should be able to run vgreduce --removemissing --force vgQ3 http://rpmfind.net/linux/RPM/centos/updates/6.6/x86_64/Packages/lvm2-2.02.111-2.el6_6.1.x86_64.html

cannot remove faulty device from LVM2 raid1 mirror

Related

Recent Posts