[Beowulf] RAID for home beowulf

[Tomislav Maric]

Tony Travis wrote:
> Tomislav Maric wrote:
>> [...]
>> I've seen Centos mentioned a lot in connection to HPC, am I making a
>> mistake with Ubuntu??
> 
> Hello, Tomislav.
> 
> [Just let me put my flame-proof trousers on...]
> 
> I know a lot of HPC people on this list use RH-based distros, but I use 
> Ubuntu for HPC and I think it's very good. In fact I started a thread on 
> the Ubuntu forums about EasyUbuntuClustering:
> 
>    http://ubuntuforums.org/showthread.php?t=1030849
> 
> I used RH6-9, and Fedora core2, but I switched to Debian and now Ubuntu.
> 

So it can be done. :) Great, I love Ubuntu. :)

>>> You also need to be aware that RAID5 is not so good when writing to the
>>> disk, because parity has to be calculated and written to the disk. In
>>> fact this performance penalty has lead to a campaign against RAID5:
>>>
>>>    http://www.baarf.com/
>> Okaay. :) There's war going on against it.
> 
> This campaign really made me think twice about what I was doing using 
> RAID5. I lied to you (a bit) because I've bought more 3ware 8006-2's to 
> put /home on RAID10 for our Beowulf servers. I must admit that hot-swap 
> is one of the main reasons, but BAARF did come into it as well.
> 
>> [...]
>> Yeah, but isn't RAID1 used for disk mirroring? How then would I get any
>> speedup? From what I've read so far, data stripping is where I get the
>> performance boost when using RAID: there's no real parallel
>> writing/seeking applied to single data stream in RAID1...
> 
> You don't get a speedup when writing, but you avoid the performance 
> penalty of writing to RAID5. Writing to a RAID1 is essentially the same 
> speed as writing to a single disk. However, you do get a performance 
> benefit when reading from RAID1, and you decouple disk access between 
> the 'system' disk and /home on the RAID1 if you follow my suggestion.
> 

OK, thanks, I think I'm getting the hang of this... I guess I'll have to
 balance some goals and play around with the configurations.

> On COTS motherboards the main bottleneck is the PCI bus anyway, not the 
> SATA disks. Have you benchmarked the disk i/o performance that your 
> hardware is capable of?
> 

I'm assembling and configuring something like this for the first time
ever. So the answer is: not yet, haven't thought of that, thank you very
much for the advice. :)

>> [...]
>> Thanks, my only problem is that I've reached my financial limits for my
>> home project so I have to work with what I have. :) I'll definitely save
>> this e-mail in my "importants" folder.
> 
> I set out with similar ideas to yours, but in the end you get what you 
> pay for. My four-disk software RAID systems work fine and they survive 
> single disk failures without crashing or losing any data. However, we've 
> had a couple of near double disk failures so I decided to put the system 
> and /backups on hardware RAID1 instead. I'm still using software RAID5 
> for /home, and I think this is a reasonable compromise between cost, HA 
> and performance.
> 

I figured it's something like that... hardware RAID will have to wait
for a while, definitely...

Thanks,
Tomislav

> Good luck!
> 
>    Tony.