July 03, 2007

Mirroring system drive under FreeBSD. Simply.

When configuring a new server, one of the first thing I do is build a RAID-1 mirror across the two hard drives that present in nearly all entry-level to mid-range servers. Good if you have a hardware RAID controller, but the cheaper models don't - they just have two fast (often SATA, not SCSI) drives to live with.

The question is - how do you mirror the system drive once the system has been installed ? Oh, did I mention it is FreeBSD ?

I used to follow the instructions here:

http://people.freebsd.org/~rse/mirror/

and it is good and correct and worked for me many times. Ralf S. Engelschall has undoubtedly put a lot of effort having the script even better over time. It essentially suggests that whenever you need to build a mirror across ad0 and ad1, you copy ad0 to ad1, then build a "mirror" with a single drive ad1 and reboot from it. After a reboot you have a "mirrored" drive ad1 as system and ad0 left aside. Then you add ad0 to the mirror, wait for it to resynchronize and reboot again. You also have to modify a few configuration files, calculate sizes of partitions etc. Such hassle is necessary because you presumably cannot add the system drive ad0 to a mirror when you have booted from it.

Unfortunately something went wrong and it didn't work with the server I had to build just recently. After the first reboot the gmirror provider failed to recognize the mirror on ad1 and boot process failed. Tried a few times - no luck. I guess the problem was in that both ad0 and ad1 have previously participated in a mirror, and I didn't have the drives cleaned as well I should, and some meta information left lurking on ad0. Anyhow, after some googling I found this other article

http://www.onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html

It manages to do the same thing with no hassle whatsoever. You simply add /dev/ad0 to mirror/gm0 and that's it. Oh, the little problem about not being able to execute gmirror label on the drive the system is currently booted from ? Easy - an undocumented (I'd presume) sysctl:

# sysctl kern.geom.debugflags=16

will rid of this nuisance. You simply set this debug sysctl and the mirror can be built right away. Somewhat an unclean solution but how much simpler.