[Beowulf] Parallel file systems

[Sabuj Pattanayek]

Gluster is the easiest clustered FS to setup vs OCFS2, GFS/GFS2,
Lustre, and XSan/Storenext.

Although, my gripe with it is that currently quotas work at the
filesystem level and not at the gluster level and this gets messy if
you've got a stripe and cluster+distributed setup across the same
filesystems on multiple storage bricks. That is, if you fill up your
quota on one node in the gluster mounted directory /dist/user
(cluster+distributed) and then try to write to /stripe/user (the
gluster stripe across all nodes), writes to the stripe will fail.
Writes to the distributed directory continue until you run out of your
quota on all the nodes.

Ideally one should setup multiple filesystems across the nodes, each
for the different types of read/write methods, but this isn't always
possible or desirable especially if you want one "global" filesystem
space.

Otherwise, performance is great using infiniband especially to the
stripes. Are there other clustered FS out there that use infi, maybe
lustre? There's also heavy development on the codebase so it's
continually evolving and improving.

HTH,
Sabuj Pattanayek

On Tue, Jan 19, 2010 at 5:26 PM, Jess Cannata <jac67 at georgetown.edu> wrote:
>
>
> On 01/13/2010 06:40 AM, tegner at renget.se wrote:
>>
>> While starting to investigating different storage solutions I came across
>> gluster (www.gluster.com). I did a search on beowulf.org and came up with
>> nothing. gpfs, pvfs and lustre on the other resulted in lots of hits.