[Beowulf] RAID for home beowulf

[Tomislav Maric]

Tony Travis wrote:
> Tomislav Maric wrote:
>> Hi everyone,
>>
>> I've finally gathered all the hardware I need for my home beowulf. I'm
>> thinking of setting up RAID 5 for the /home partition (that's where my
>> simulation data will be and RAID 1 for the system / partitions without
>> the /boot.
>>
>> 1) Does this sound reasonable?
> 
> Hello, Tomislav.
> 
> So far so good, but RAID5 trades capacity for perfomance...
> 
>> 2) I want to put the /home at the beginning of the disks go get faster
>> write/seek speeds, if the partitions are the same, software RAID doesn't
>> care where they are?
> 
> Actually, it does - Read about the 'stride' of an ext3 filesystem:
> 
>    http://wiki.centos.org/HowTos/Disk_Optimization

OK, thanks, nice to know, I've been reading about it in Tannenbaum's
book (this home cluster stuff really made me widen my horizon :) ).

I've seen Centos mentioned a lot in connection to HPC, am I making a
mistake with Ubuntu??

> 
> You also need to be aware that RAID5 is not so good when writing to the 
> disk, because parity has to be calculated and written to the disk. In 
> fact this performance penalty has lead to a campaign against RAID5:
> 
>    http://www.baarf.com/

Okaay. :) There's war going on against it.

> 
>> 3) I'll leave the /boot partition on one of the 3 disks and it will NOT
>> be included in the RAID array, is this ok?
> 
> I think you'd be better off putting your system one one of your three 
> disks, and making a RAID1 for /home from the other two. This will give 
> you a perfomance gain because RAID1 writes do not involve generating 
> parity, and you will decouple disk access between 'system' and /home. 
> You can backup your system disk to the RAID1, or reinstall if it fails.

Yeah, but isn't RAID1 used for disk mirroring? How then would I get any
speedup? From what I've read so far, data stripping is where I get the
performance boost when using RAID: there's no real parallel
writing/seeking applied to single data stream in RAID1...

>> 4) I've read about setting up parallel swaping via priority given to
>> swap partitions in fstab, but also how it would be ok to create RAID 1
>> array of swap partitions for the HA of the cluster. What should I choose?
> 
> You're going to have to decide between perfomance and HA: Sorry, you 
> can't have both with only three disks. I've built systems with four SATA 
> disks each with five partitions: /boot on ext2, swap on RAID1, / and 
> /home on RAID5, /backup on RAID5 for much the same reasons you are 
> considering doing it too. It works, and you do get HA, but performance 
> is not good. I'm now simplifying and upgrading the systems by fitting a 
> 3ware 8006-2 hardware RAID1 controller and two extra disks for /, swap 
> and /backups. I'm using the original four disks with a single partition 
> now as a software RAID5 for an ext3 /home filesystem using appropriate 
> 'stride' and directory indexing.
> 
>> I've gone through all the software raid how-tos, FAQs and similar, but
>> they are not quite new (date at least 3 years) and there's no reference
>> to clusters. Any pointers regarding this?
> 
> One thing you need to bear in mind in relation to HA is that software 
> RAID does not support hot-swap - That's why I chose the 3ware 8006-2, 
> which is not very expensive. It doesn't automatically detect new disks 
> and rebuild the RAID, but it does support hot-swap and has a web GUI:
> 
>    http://www.3ware.com/products/serial_ata8000.asp
> 

Thanks, my only problem is that I've reached my financial limits for my
home project so I have to work with what I have. :) I'll definitely save
this e-mail in my "importants" folder.

Best,
Tomislav

> Bye,
> 
>    Tony.