mdadm+zfs vs mdadm+lvm

Solution 1:

If you go for the MDADM raid6 + ZFS solution, you will lose self healing as ZFS have no data about how to recalculate/restore damaged data blocks, unless if you set copies=2 or more(which will use twice as much storage). Your ZFS storage will only report data inconsistencies.

That means if a data block cannot be properly read, ZFS will tell you which files are affected. Your whole file system shouldn't be in danger as meta data is dual redundant and can be self healed if any inconsistencies are detected.

You can also grow your ZFS file system by adding more vdevs to your pool, which will be dynamically striped. For example you can create a new raidz2 vdev with 4 to 8 disks and add it to your pool. You can even add a mirrored vdev to it.