[Beowulf] Parallel File system as a storage Solution

[Imran Khan]

Hello

Have you guys tried TerraGrid ?

-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org] On
Behalf Of Tim Cullen
Sent: 15 August 2005 18:26
To: Mark Hahn; Robert Latham
Cc: beowulf at beowulf.org
Subject: Re: [Beowulf] Parallel File system as a storage Solution

Don't know if this will help you or not, but as far as
parameterizing Lustre goes. You can use the lfs
utility to specify how a single file or all files in a
given directory are laid out across your storage
servers. 

tim

--- Mark Hahn <hahn at physics.mcmaster.ca> wrote:

> > I'm one of the PVFS2 developers -- just warning
> you that I might be a
> > bit biased :>
> 
> I wonder if you could offer some comments on how you
> see PVFS2 vs Lustre.
> 
> > Typical /home directory usage patterns are quite
> different from
> > typical other file systems.  Users have lots of
> small files
> > (.dotfiles, directories, mail (if it's in maildir
> or mh format),
> > source code).  They run 'ls' (so end up calling
> stat(2) all the time.  
> 
> yes - we actually force users to organize their
> files such that 
> they only get a fairly small /home quota (~200MB),
> but tons of 
> other space (/work (which is shared cross-cluster),
> /scratch, 
> node-local /tmp, etc).  it's also worth pointing out
> that even the 
> underlying raid structure for these filesystems
> should really be 
> different: /home has fairly small blocks, for
> instance, so a big-stripe
> raid5 is really a bad bet.  but /scratch can be
> raid0, and /work something
> like raid 50...
> 
> or that /home makes effective use of the pagecache,
> and /scratch is 
> basically write-through.
> 
> > I think you're on the right track here:  have an
> NFS-hosted /home
> > directory where users can compile their
> applications.  have a pvfs2
> > /scratch file system where those applications can
> write their IO when
> > they run.
> 
> hmm.  it would be nice if FS's like Lustre and PVFS
> could be parameterized to
> provide adequate /home performance as well, since
> it's not nearly as easy
> to scale NFS.
> 
> anyone using fscache to speed up NFS in an HPC
> environment (even just 
> for homes)?
> 
> thanks, mark hahn.
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or
> unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
> 



		
____________________________________________________
Start your day with Yahoo! - make it your home page 
http://www.yahoo.com/r/hs 
 
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf