[Beowulf] RAID for home beowulf

[Tony Travis]

Mark Hahn wrote:
>[...]
> bah.
> redundancy costs - your only choice is what kind of redundancy you want.
> raid5 has its issues, but really what's needed is redundancy managed
> by the filesystem.  doing it block-level is what introduces the pain.

Hello, Mark.

You're right ;-)

I've just been reading about the Hadoop file system. That recommends not 
using RAID at all, and just relying on configurable levels of redundancy 
in the filesystem. However, I've got a complete mental block about using 
the Hadoop filesystem 'shell'. I'll persevere, though because it does 
look very interesting. I've been trying to learn how to use it for ages!

>>> 3) I'll leave the /boot partition on one of the 3 disks and it will NOT
>>> be included in the RAID array, is this ok?
>> I think you'd be better off putting your system one one of your three disks,
>> and making a RAID1 for /home from the other two. This will give you a
> 
> I disagree.  for one, I strongly advice against partition-o-philia:
> the somewhat traditional practice of putting lots of separate partitions
> on a system.  but for 3 disks, linux MD has a nice mode where each block
> is stored twice.  so you get 2/3 space efficiency and potentially better
> performance.

I'm not quite sure I understand what you're saying here: I have my 
system organised in such a way that my 3ware RAID's provide a bare-metal 
recovery backup using just three partitions:

   sda1 /        40GB
   sda2 swap      8GB
   sda3 /backup  rest of disk

The 'md' RAID5 is four disks with a single partition:

   sd[abcd]1     RAID autodetect

How is this "partition-o-philia"?

>[...]
> MD software raid certainly can support HS - it's mainly a feature of the
> controller.  (this is not to say that most controllers do HS well.)

I'm interested in that - It was only alpha release when I looked, and 
not recommended for production use. I did some experiments, and settled 
on hot-swap using the 3ware 8006-2's, with 'md' software RAID10.
.
Bye,

   Tony
-- 
Dr. A.J.Travis, University of Aberdeen, Rowett Institute of Nutrition
and Health, Greenburn Road, Bucksburn, Aberdeen AB21 9SB, Scotland, UK
tel +44(0)1224 712751, fax +44(0)1224 716687, http://www.rowett.ac.uk
mailto:a.travis at abdn.ac.uk, http://bioinformatics.rri.sari.ac.uk/~ajt