LVM2 vs MDADM performance
I've used MDADM + LVM2 on many boxes for quite a while. MDADM was serving for both RAID0 and RAID1 arrays, while LVM2 where used for logical volumes on top of MDADM.
Recently I've found that LVM2 could be used w/o MDADM (thus minus one layer, as the result - less overhead) for both mirroring and stripping.
However, some guys claims that READ PERFORMANCE on LVM2 for mirrored array is not that fast as for LVM2 (linear) on top of MDADM (RAID1) as LVM2 does not read from 2+ devices at a time, but use 2nd and higher devices in case of 1st device failure. MDADM reads from 2 devices at a time (even in mirrored mode).
Who could confirm that?
Solution 1:
I'd bet that not even the LVM authors use LVM's RAID facilities. MD is a lot more efficient, mature and complete; and has more development dedicated to it.
The 'less layers - less overhead' is frequently not true; even if the CPU could take a little longer to get to disk, this would be totally overcomed by any small disk-related improvement of MD, of which are a lot.
Solution 2:
I've tinkered with LVM2's mirror support, and I can say: It's not really meant to replace RAID1.
The real use for LVM2 mirroring is to transfer data between volumes. Say you have a drive failing, and you want to get data from point A (which is in peril) to point B (which is safe). The point of the LVM2 mirror function is to clone off the data to other parts automatically, while allowing regular I/O to proceed. After the "mirror" is caught up, you break the mirror and remount your data on the new, safe location.
The speed at which it does this is less-than-stellar. Like, worse than 50% slower than just a straight RAID1. In fact, it's so slow I can watch two drives that are part of an LVM2 mirror strobe the activity light at different times. But if you need to shift data between physical locations, it'll do the job transparently, and that's what LVM's really all about - transparent management of the storage layer while the filesystem is active. RAID is more about avoiding data loss due to a single point of hardware failure.
The issue of "overhead" really isn't there. The only real issue you'll encounter is recovery, and that's a posting unto itself. Recovering data from a blown filesystem is hard, recovering it from a three-layer filesystem (RAID/LVM/Ext4) is a PITA. So it's kinda important to make sure the drives are healthy (SMART), the array is healthy (mdadm), your volume groups are healthy (LVM2), and the filesystem is healthy (fsck). I've lived through this once, and I would rather not do it again.