[Beowulf] Parallel file systems

[Sabuj Pattanayek]

On Thu, Jan 21, 2010 at 4:15 PM, Mark Hahn <hahn at mcmaster.ca> wrote:
>> Otherwise, performance is great using infiniband especially to the
>> stripes.
>
> I find that people have greatly differing expectations, so what does "great"
> mean to you?

The stripe is across 5 machines and I'm getting about 3 times speedup
on writes and 2.5 times speedup on reads vs the same tests going
directly to the XFS filesystem on a single node. Not hitting anything
close to the peak interconnect bandwidth, about 4gbps on writes and
7gbps on reads.

>
> I would expect to achieve ~90% of the peak bandwidth of the interconnect,
> for instance, assuming enough and fast-enough storage targets.  this is easy
> enough to achieve using  Lustre in my experience (albeit I haven't really
> tried eg QDR IB.)

What about your setup above? What sort of speedup are you getting vs a
single node, assuming your (storage) nodes are homogeneous in terms of
hardware.