How to add disk back to RAID and replace removed
Received a message that RAID went into degraded state:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
524224 blocks [2/2] [UU]
md2 : active raid10 sda5[0] sdb5[1] sdd5[3] sdc5[2](F)
1128864768 blocks super 1.2 512K chunks 2 near-copies [4/3] [UU_U]
md1 : active raid10 sda2[0] sdb2[1] sdc2[2] sdd2[3]
41909248 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
unused devices: <none>
Logged in to the server and removed fail disk (didn't mark it as failed):
#> mdadm --manage /dev/md2 --remove /dev/sdc5
Here is current status:
#> cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sda1[0] sdb1[1]
524224 blocks [2/2] [UU]
md2 : active raid10 sda5[0] sdb5[1] sdd5[3]
1128864768 blocks super 1.2 512K chunks 2 near-copies [4/3] [UU_U]
md1 : active raid10 sda2[0] sdb2[1] sdc2[2] sdd2[3]
41909248 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
unused devices: <none>
And more details:
#> mdadm --detail /dev/md2
/dev/md2:
Version : 1.2
Creation Time : Sun Apr 6 13:42:43 2014
Raid Level : raid10
Array Size : 1128864768 (1076.57 GiB 1155.96 GB)
Used Dev Size : 564432384 (538.28 GiB 577.98 GB)
Raid Devices : 4
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Wed Jun 3 10:29:14 2015
State : active, degraded
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : near=2
Chunk Size : 512K
Name : yyc-controller:2
UUID : c9fe371a:2c8345bf:68054eba:bb4d78aa
Events : 41545
Number Major Minor RaidDevice State
0 8 5 0 active sync /dev/sda5
1 8 21 1 active sync /dev/sdb5
2 0 0 2 removed
3 8 53 3 active sync /dev/sdd5
Now I'm trying add it back but it fails:
#> mdadm --manage /dev/md2 --add /dev/sdc5
mdadm: add new device failed for /dev/sdc5 as 4: Invalid argument
How can I add disk as Number 2 instead of removed?
I had a similar issue. One disk of a raid1 had some issues and I had removed the entire disk from the raid. After a destructive badblocks check writing random patterns, three of the four partitions of the disk could be added back to the raid without problems. Only the last partition failed with the above error. syslog reported:
md: sdb4 does not have a valid v1.2 superblock, not importing!
I didn't want to stop the raid and completely rebuild it, as sometimes suggested, but I could fix this by zeroing out the random patterns, where mdadm seemd to find some superblock like bytes using dd:
dd of=/dev/sdb4 if=/dev/zero bs=1M count=1
Afterwards mdadm could add the partition without errors.