[Beowulf] Network Filesystems performance

[Buccaneer for Hire.]

--- Chris Samuel <csamuel at vpac.org> wrote:

> On Fri, 24 Aug 2007, Michael Will wrote:
> 
> > I tested several NFS server configurations with a
> 19 node cluster.
> 
> Same, but with 150+ nodes and about 500+ CPUs.
> 
> > The first advise is to stay away from redhat for
> file servers since
> > they have some bursty I/O bugs and don't support
> XFS.
> 
> Amen.
> 
> We've run through Fedora (OK, but release cycle too
> quick), Ubuntu 
> (Dapper has longer support, but a bit flakey) and
> now run Debian Etch 
> (very happy).
> 
> Fedora & Ubuntu were on a pair of IBM e325 with
> 2x2.0GHz Opterons and 
> gigabit ethernet connected through to each side of
> an Apple XServe 
> RAID array.
> 
> Debian is running on an IBM x3550 with a pair of
> 3GHz woodcrests and a 
> couple of bonded gigabit ethernets connected through
> via SAS to some 
> external SATA EXP3000 arrays.  We decided to use the
> hardware RAID to 
> make smaller individual RAID5 arrays and then stripe
> across them 
> using the Linux MD driver.  Seems to work rather
> nicely!

A stock kernel will from Redhat will not give you the
performance you need.  We use Fedora and
patch/configure a standard kernel from kernel.org.  
On the back end, we are currently using Clariions
(3-80) and build smaller LUNs per EMC recommendations
and metaLUN them.  That really helps performance. Our
industry is know for humongous data files.


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com