[Beowulf] [tjrc@sanger.ac.uk: Re: [Bioclusters] topbiocluster.org]

[Guy Coates]

>
> I guess that we've had the opposite experience with GPFS.  We have
> that file system on Linux x86 in an NSD configuration, with two
> servers attached to a SAN distributing the the file system to about
> fifty execution nodes over gigabit ethernet.

The trick with GPFS is lots of luns. GPFS parallelises IO at the lun
level, so the more luns, the greater the potential for parallel IO. Our
config has 28 nodes with local IDE disks configured as NSDs, pooled into a
single filesystem.

We have some SAN storage configured as GPFS, and initially, got really
poor performance as we had just configured two large Luns. By breaking the
storage up into smaller luns we got better numbers.  There are other
gotchas for SAN storage, such as tuning the raidset size to the filesystem
block size, but that is all well documented in the GPFS tuning-guide docs
from IBM.

Cheers,

Guy


This cluster run
> bioinformatics applications - lots of BLAST.  Concurrent BLAST jobs
> can run quite slowly reading the databases from GPFS.  Just yesterday
> someone ran BLAST accross twenty-five nodes in that fashion, and the
> individual processes shambled along, barely using more than 15% of
> the CPU.  Meanwhile the NSD servers were showing loads of around
> twenty, and the GPFS file was annoyingly unresponsive in interactive
> use.  MEGABLAST is even worse.  The folks around here have given up
> on running concurrent MEGABLASTs in GPFS, and instead first stage the
> databases they need to local disk on the execution hosts.
>
> A large part of the problem could be the SATA disks in the SAN, but
> that's what we have to work with.  We're vaguely casting about for
> alternatives to GPFS.  One study I've found comparing cluster or
> parallel file systems
>
> 	http://www.linuxclustersinstitute.org/Linux-HPC-Revolution/Archive/PDF05/17-Oberg_M.pdf
>
> indicates that alternative aren't very much better.
>
> David S.
>
> >
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>

-- 
Dr. Guy Coates,  Informatics System Group
The Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1HH, UK
Tel: +44 (0)1223 834244 x 6925
Fax: +44 (0)1223 494919