[Beowulf] High Performance for Large Database

[Andrew Piskorski]

On Tue, Nov 09, 2004 at 03:41:52PM +1100, Felix Rauch Valenti wrote:
> On Wed, 27 Oct 2004 09:29:58 +0800, Laurence Liew
> <laurenceliew at yahoo.com.sg> wrote:
> [...]
> > 3. Try running Postgresql on a cluster filesystem like PVFS - it is not
> > gauranteed as it probably fails the ACID test for a SQL compliant
> > database. The basic idea is that if we cannot parallelise the database -

The term I've heard for this is "full POSIX semantics".

POSIX specifies how filesystems are "supposed" to work and what
guarantees they are supposed to provide, and your typical RDBMS
expects (nay, REQUIRES) the underlying filesystem to fully comply to
POSIX.  If it doesn't, Bad Things happen.

All the usual direct-attached Unix and Linux file systems do have such
POSIX semantics.  NFS does NOT, and most other cluster filesystems
also do not - or so I've heard.  Lustre explicitly claims that it DOES
have full POSIX semantics.

  http://www.lustre.org/
  http://openacs.org/forums/message-view?message_id=128060
  http://www.lustre.org/docs/lustre-datasheet.pdf

Some cluster fileystems folks believe that full POSIX semantics are
neither necessary nor - for performance reasons - desirable in cluster
filesystems, and are trying to figure out what more relaxed set or
sets of requirements will best meet the needs of their parallel HPC
applications.

ClusterWorld magazine is currently running a series of articles on
cluster filesystems, but so far (the Oct. and Nov. issues), they've
completely failed to provide any real description or classification of
filesystems according to what semantics they provide, nor talked at
all about the significance of filesystem semantics for applictions
trying to use those filesystems.  Does anyone know of a source that
does nicely provide that info?

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/