[Beowulf] RAID for home beowulf

[Tony Travis]

Tomislav Maric wrote:
> [...]
> I've seen Centos mentioned a lot in connection to HPC, am I making a
> mistake with Ubuntu??

Hello, Tomislav.

[Just let me put my flame-proof trousers on...]

I know a lot of HPC people on this list use RH-based distros, but I use 
Ubuntu for HPC and I think it's very good. In fact I started a thread on 
the Ubuntu forums about EasyUbuntuClustering:

   http://ubuntuforums.org/showthread.php?t=1030849

I used RH6-9, and Fedora core2, but I switched to Debian and now Ubuntu.

>> You also need to be aware that RAID5 is not so good when writing to the
>> disk, because parity has to be calculated and written to the disk. In
>> fact this performance penalty has lead to a campaign against RAID5:
>>
>>    http://www.baarf.com/
> 
> Okaay. :) There's war going on against it.

This campaign really made me think twice about what I was doing using 
RAID5. I lied to you (a bit) because I've bought more 3ware 8006-2's to 
put /home on RAID10 for our Beowulf servers. I must admit that hot-swap 
is one of the main reasons, but BAARF did come into it as well.

>[...]
> Yeah, but isn't RAID1 used for disk mirroring? How then would I get any
> speedup? From what I've read so far, data stripping is where I get the
> performance boost when using RAID: there's no real parallel
> writing/seeking applied to single data stream in RAID1...

You don't get a speedup when writing, but you avoid the performance 
penalty of writing to RAID5. Writing to a RAID1 is essentially the same 
speed as writing to a single disk. However, you do get a performance 
benefit when reading from RAID1, and you decouple disk access between 
the 'system' disk and /home on the RAID1 if you follow my suggestion.

On COTS motherboards the main bottleneck is the PCI bus anyway, not the 
SATA disks. Have you benchmarked the disk i/o performance that your 
hardware is capable of?

> [...]
> Thanks, my only problem is that I've reached my financial limits for my
> home project so I have to work with what I have. :) I'll definitely save
> this e-mail in my "importants" folder.

I set out with similar ideas to yours, but in the end you get what you 
pay for. My four-disk software RAID systems work fine and they survive 
single disk failures without crashing or losing any data. However, we've 
had a couple of near double disk failures so I decided to put the system 
and /backups on hardware RAID1 instead. I'm still using software RAID5 
for /home, and I think this is a reasonable compromise between cost, HA 
and performance.

Good luck!

   Tony.
-- 
Dr. A.J.Travis, University of Aberdeen, Rowett Institute of Nutrition
and Health, Greenburn Road, Bucksburn, Aberdeen AB21 9SB, Scotland, UK
tel +44(0)1224 712751, fax +44(0)1224 716687, http://www.rowett.ac.uk
mailto:a.travis at abdn.ac.uk, http://bioinformatics.rri.sari.ac.uk/~ajt