Recovering as much as possible from a LVM where some PVs failed

I have an LVM drive & filesystem made up of 4 disks. One of them has had a hardware failure and doesn't work anymore. When this happeend some of the files were readable on the filesystem, and some gave errors. I have rebooted the machine and now the (broken) filesystem (it's ext4) doesn't mount at all. I have run fsck/e2fsck on the device (/dev/volgroupname/lvname). I have taken out the bad disk, and put in a brand new disk of the same capacity, and add that as PVs to the VG.

Is there anyway I can recover as much data as possible from it?

Each of the PVs was ~500GB, so maybe there's some chunks of data there that I can recover?

(Yes I know I should have backups, or use a real raid set up, but these files aren't critical, but I'd like to get as many as possible back)

mount reports:

mount: wrong fs type, bad option, bad superblock on /dev/mapper/volgroup_lvname,
       missing codepage or helper program, or other error

with dmesg:

[62004.117164] EXT4-fs (dm-9): bad geometry: block count 967555072 exceeds size of device (478894080 blocks)

Solution 1:

I strongly suggest using ddrescue to take a copy of each component disk and running testdisk/photorec on the copied images.

Alternatively you can try to: - mount the filesystem via a backup superblock - use fsck.ext4 to repair the filesystem. Note: this will not really repair the filesystem, as an entire device failed. However it has a chance to repair the superblock and some directory metadata.

Anyway, fiddling with a broken filesystem is a dangerous game as you risk to amplify the already existing corruption. Hence my suggestion to take a binary image of each disk.