RAID1 Woe
Here’s an annoying thing. I have this box which is used as a development database by one of our dev teams. It’s a lightweight little thing – an HP ML110 quad-core jobbie with a couple of 500GB SATA drives configured as RAID1 using software RAID under CentOS Linux. The thing runs Oracle9i in 4GB of RAM, so it’s quite busy.
All fine and dandy. Or at least it was. About a week after it was deployed in the dev environment, one of the disks dropped out of the RAID array. "Odd," thought I. I hot-re-added the disk to the array and it rebuilt quite happily and passed an fsck just fine. Nothing untoward in the system logs either.
A couple of days later, it did it again but this time was accompanied by a lengthy vomit of SMART errors in the logs. Not so good. So, I shut the box down and slapped a third disk in there – the intention being to add it to the array as a spare so that when the dodgy disk DID fail, it would swap in. While I was in there I checked the SATA cables to make sure everything was secure – all seemed fine (although I confess I did move one of the disks from SATA4 to SATA2 for the sake of neatness).
Partitioned the new disk, added it to the array, and here’s where the wierdness begins.
First off, catting /proc/mdstat lists the partitions is being part of the array, but the drive status is [UU]. I’d have expected it to be [UUS] or similar. Secondly, the dodgy disk is now stubbornly refusing to fail again. It’s possible that it was just a bad SATA cable connection, but I really want the bugger to fail so I know if adding a spare to a RAID1 array will rebuild automatically or not.
Tres annoying.
Leave a Comment