[Beowulf] PVFS or NFS in a Beowulf cluster?

[Robert G. Brown]

On Mon, 17 Jan 2005, Kumaran Rajaram wrote:

> 
>    I would suggest PVFS2. It offers greater bandwidth (proportional to
> number of I/O nodes dedicated) compared to NFS. Also, you may use PVFS2
> native file interface (rather than POSIX I/O interface) which offers
> flexible parallel I/O interface for scientific workloads. Only caveat, is
> fault-tolerance support and you might need to employ either
> software/hardware RAID for disk failures and heartbeat mechanism for
> server failures. If the environment has less probability of
> server/disk/netowrk faults, then PVFS2 is a good choice for scientific
> workloads and parallel applications.

Hmmm, I'd have recommended the usual "it depends".  If all you are doing
is starting jobs from home directories or mounted project space and
writing occasional results back to same (not exactly hammering the
disks) and your storage needs are modest -- within the means of a single
RAID 5 server (say ~1 TB or less, these days) I'd personally advise
sticking with tried-and-true NFS.

NFS has all sorts of well-known and in some cases venerable "problems",
but it has also been the absolute bulwark of client-server computing in
unix environments for some 20 years, the sine qua non of the workstation
LAN.  Most of its problems are manageable and invisible under ordinary
usage.

Only if your needs are extreme -- needs for a LARGE filespace, for lots
of data parallelism, etc -- would I recommend looking into a non-NFS
solution, especially for a first cluster of very modest size.

I mean, disk is SO cheap at less than $1/GB.  I have more disk in EACH
of my home computers than existed in the world when NFS was invented.
Setting up a RAID server for a small cluster with 100's of GB costs a
total of well under $1000 -- so little you can get two and use one to
back up the other, if you'd rather do that than mess with a tape backup
unit (which will cost quite a bit MORE than $1000).  Setting up NFS on a
server takes a few minutes.  It would take longer just to find PVFS and
read the documentation on HOW to install it...

If NFS doesn't work for you, then sure, look beyond it.  But start out
simple, especially if you are already familiar with NFS servers and
clients (likely) and not so familiar with PVFS (also likely).

   rgb

> 
> -Kums
> 
> On Thu, 13 Jan 2005, Stavros E. Zogas wrote:
> 
> > Hi at all
> > I intend to setup a beowulf cluster(16+ nodes) for scientific
> > applications(fortran compilers) in a University department.What am i supposd
> > to use???PVFS or NFS for file system??
> > Stavros
> >
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu