Need to find which drives are mirrored within RAID-10 array
I have a home media server running on openSUSE 12.2. I'm using eight 2TB drives in a RAID-10 configuration. I deliberately bought two different types of drives: four Seagate Barracuda Green and four Western Digital Red. My goal is to configure the RAID such that each mirrored pair within the array will consist of dissimilar drives (ie, one Seagate drive and one WD drive). YaST2 Partitioner unfortunately did not give me this level of choice in the structure of the array, so now I'm trying to find out what the default RAID10 structure looks like.
I do know the following:
- sdc, sdd, sde, and sdf are all WD drives
- sdg, sdh, sdi, and sdj are all Seagate drives
I chose the default 'n2' layout when creating the RAID. My guess based upon info from these two sources is that adjacent drives are mirrored (ie, sdc==sdd, sde==sdf, etc), but I want to know for sure:
- http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10
- https://serverfault.com/questions/200725/mirrored-and-stripped-hd-drives-in-raid10
Here is the output of 'mdadm --detail /dev/md0':
/dev/md0: Version : 1.0 Creation Time : Sat Mar 16 12:55:11 2013 Raid Level : raid10 Array Size : 7814045696 (7452.05 GiB 8001.58 GB) Used Dev Size : 1953511424 (1863.01 GiB 2000.40 GB) Raid Devices : 8 Total Devices : 8 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Sat Mar 16 13:09:37 2013 State : active, resyncing Active Devices : 8 Working Devices : 8 Failed Devices : 0 Spare Devices : 0 Layout : near=2 Chunk Size : 2048K Resync Status : 1% complete Name : aldaris:0 (local to host aldaris) UUID : c6cc3943:97394500:b77d44cd:f02ed876 Events : 149 Number Major Minor RaidDevice State 0 8 33 0 active sync /dev/sdc1 1 8 49 1 active sync /dev/sdd1 2 8 65 2 active sync /dev/sde1 3 8 81 3 active sync /dev/sdf1 4 8 97 4 active sync /dev/sdg1 5 8 113 5 active sync /dev/sdh1 6 8 129 6 active sync /dev/sdi1 7 8 145 7 active sync /dev/sdj1
And here are the contents of /proc/mdstat:
Personalities : [raid10] md0 : active raid10 sdj1[7] sdi1[6] sdh1[5] sdg1[4] sdf1[3] sde1[2] sdd1[1] sdc1[0]
7814045696 blocks super 1.0 2048K chunks 2 near-copies [8/8] [UUUUUUUU]
[>....................] resync = 4.8% (375163456/7814045696) finish=1206.5min speed=102751K/sec
bitmap: 57/59 pages [228KB], 65536KB chunk
unused devices: <none>
So my questions are:
- How do I tell which drives are mirrors of each other?
- Is there a way to change this, or should I just swap the wires around (since that will swap the drive letters) and then rebuild the RAID?
Thanks in advance.
Tangential note, for anyone wants to know my reasoning for doing this is: Drives of the same model and batch, operated under similar usage loads, uptime, and temperature have little systematic variation, and differences in time to failure between drives will be primarily driven by random variation in the manufacturing process. This increases the risk of multiple drives dying at once. By purchasing drives not just from different batches but completely different manufacturers, I am introducing systematic variation into my array, thus influencing which drives will fail at similar times.
Recent versions of mdadm
show this right in the details of the array. Example from mdadm v3.3 - 3rd September 2013
$ mdadm --detail /dev/md1
/dev/md1:
Version : 1.1
Creation Time : Tue Aug 23 11:45:41 2016
Raid Level : raid10
Array Size : 3864803328 (3685.76 GiB 3957.56 GB)
Used Dev Size : 1932401664 (1842.88 GiB 1978.78 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Fri Aug 26 09:39:28 2016
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : near=2
Chunk Size : 512K
Name : px4-300r-THXOAP:1 (local to host px4-300r-THXOAP)
UUID : 5ee06437:83dfdb64:808feaa2:5d57b1e6
Events : 620
Number Major Minor RaidDevice State
4 8 50 0 active sync set-A /dev/sdd2
1 8 34 1 active sync set-B /dev/sdc2
2 8 18 2 active sync set-A /dev/sdb2
3 8 2 3 active sync set-B /dev/sda2
Note the denotation set-A
or set-B
. In the above case, sdd
and sdb
can fail together without data loss.
It is possible this data is not available while the array is rebuilding though.
I had the same issue and after googling a while I didn't find a reliable answer. After giving it some thoughts, I figured that the mirrors have the same data and so we could compare some part of it.
NOTE: BE CAREFUL, IF YOU HAVE MORE THAN 2 DRIVES WITH THE SAME CHECKSUM YOU ARE PROBABLY COMPARING EMPTY DISKSPACE, CHOOSE ANOTHER OFFSET (skip option).
With this few commands, you can figure it out:
for disk in sda sdb sdc sdd
do
echo -n "$disk = ";
dd if=/dev/$disk skip=1M bs=1M count=1 2>/dev/null | md5sum;
done
This will output something like:
sda = 7c4ef0f3e0143b35e044d5d65908a3a2 -
sdb = 7c4ef0f3e0143b35e044d5d65908a3a2 -
sdc = e02f7d61ad3791bd691da5b7516928a5 -
sdd = e02f7d61ad3791bd691da5b7516928a5 -
Now we know that sda/sdb is one mirror and sdc/sdd another one. One of each must stay to avoid data loss.
The "dd" command is reading one time (count=1) one Megabyte (bs=1M) at one Megabyte offset from the disk start (skip=1M). Don't skip=0, because the begining of the disk contains different information. The data usually begins after 1MB.