Creating a partitioned raid1 array for booting a debian squeeze system

I'd like to have the following raid1 (mirror) setup:

/dev/md0 consists of /dev/sda and /dev/sdb

I created this raid1 device using

mdadm --create --verbose /dev/md0 --auto=yes --level=1 --raid-devices=2 /dev/sda /dev/sdb

This gave a warning about metadata being 1.2 and my system might not boot. I cannot use 0.9 because it restricts the size of the raid to 2TB and I assume grub shipped with latest debian (squeeze) should be able to handle metadata 1.2.

So then I created the needed partitions like this:

# creating new label (partition table)
parted -s /dev/md0 mklabel 'msdos'

# creating partitions
sfdisk -uM /dev/md0 << EOF
0,4096
,1024,S
;
EOF

# making root filesystem
mkfs -t ext4 -L boot -m 0 /dev/md0p1

# making swap filesystem
mkswap /dev/md0p2

# making data filesystem
mkfs -t ext4 -L data /dev/md0p3

Then I mounted the root partition, copied a minimal debian install inside and temporary mounted /dev /proc /sys. Afer this I chrooted to the new root folder and executed:

grub-install --no-floppy --recheck /dev/md0

However this fails badly with:

/usr/sbin/grub-probe: error: unknown filesystem. Auto-detection of a filesystem of /dev/md0p1 failed. Please report this together with the output of "/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs -v /boot/grub" to

I don't think it's a bug in grub (so I didn't report it yet) but a fault of mine. So I really wonder how to properly setup my raid1, everything I tried so far failed.

Solution 1:

You should install grub on your disks MBR, not on /dev/md0, so run grub-install /dev/sda ; grub-install /dev/sdb. Is this work better ?

Solution 2:

I don't think it's a good idea to create partitions on the raid. A better approach would be to create partitions on each device and then create the raids accordingly.

In addition create 2 bootable partitions of exactly the same size (just as the raid partitions should match in size) which will not be part of the raid and make sure they're exactly the same so the system can boot off of either one.

I also prefer separate partitions for the usual locations such as /var, /tmp, /usr, /home etc.

For example:

/dev/sd[ab]1 - /boot
/dev/sd[ab]2 - swap
/dev/sd[ab]3 - /
/dev/sda[ab]4 - /usr
/dev/sda[ab]5 - /tmp
/dev/sda[ab]6 - /var
/dev/sda[ab]7 - /home

Create raids out of each partition except the first. For example:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[ab]2

and so forth.

Then assign /boot to /dev/sda1 and make sure the MBR is saved to both /dev/sda and /dev/sdb. Once the whole system has been installed you do (bs=500M will speed up dd a lot, adjust 500M to about 2/3 of your system's memory):

dd bs=500M if=/dev/sda1 of=/dev/sdb1

Now as long as the bios will try to boot from either disk then in case one of the disks fails the system will automagically boot from the other disk because the UUIDs are exactly the same. Don't forget to sometimes repeat the dd command if /boot has changed, say with a kernel upgrade.

You do want to test it though by changing the bios boot priority. And if you feel lucky test it by yanking out one disk :-)

This will work with a raid10 also, I have done it with both a raid1 and raid10, even a 3 disk raid10 (which is actually possible with mdadm).

By the way you have to use a boot partition that's not part of the raid because otherwise your system can't boot. There has to be a way for the raid to be started and since it is a softraid the kernel first has to be loaded in order for the raid to be recongised.