[Beowulf] large scratch space on cluster

[Joe Landman]

Hearns, John wrote:
> I was wondering if somebody could help me here a bit.
> For some of the calculations we are running on our cluster we need a 
> significant amount of disc space. The last calculation crashed as the ~700 GB 
> which I made available were not enough. So, I want to set up a RAID0 on one 8 
> core node with 2 1.5 TB discs. So far, so good.
> 
> 
> Sounds like a cluster I might have had something to do with in a past life...
> 
> 
> 700 gbytes! My advice - look closely at your software and see why it needs this scratch space,
> and what you can do to cut down on this.

Heh... some of the coupled cluster GAMESS tests we have seen/run have 
used this much or more in scratch space.

Single threaded readers/writers ... you either need a very fast IO 
device, or like John suggested, you need to examine what is getting 
read/written.

700GB @ 1GB/s takes 700 seconds, roughly 11m40s +/- some.
700GB @ 0.1GB/s takes 7000 seconds, roughly 116m40s +/- some (~2 hours).

A RAID0 stripe of two drives off the motherboard will be closer to the 
second than the first ...

> 
> Also, let us know what code this is please.
> You're right about network transfer of scratch files like that - if at all possible,
> you should aim to use local scratch space on the nodes.
> $VENDOR (I think in Warwick!) should be very happy to help you there!

I know those guys! (and they are good).

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615