[Beowulf] PVFS on 80 proc (40 node) cluster

[Jeff Candy]

Jeff:

>>Does anyone have experience with PVFS on a cluster
>>in the range of 80 processors (40 dual nodes with
>>gigE)?
>>
>>I am considering this over the usual NFS-master
>>node stup since we expect to multiple users/jobs
>>running concurrently.

Reuti:

> on the one hand it sounds interesting. I would fear that in a cluster (where 
> each node should do heavy calculations and use the own disk for local scratch 
> data) the performance will be worse than a dedicated file server with a RAID. 
> What programs will your cluster run and how are the users submitting the jobs? 

- the program is a large physics code that does I/O
(200KB or less) every 10 to 60 sec.  Every 10min or
so, a 100MB file is written.

- users will submit with PBS (typically, I expect <= 3
jobs to run concurrently).

- I want a *single* filesystem, so no local scratch
will be used.

Are you in favour of a single master with a RAID
filesystem, NFS mounted by all nodes?  I wonder
what fraction of systems now use this scheme.

Thanks for your input.

Jeff