[Beowulf] High Performance for Large Database

[Laurence Liew]

Hi,

You may wish to search thru the beowulf list or google for "beowulf and 
databases and postgresql"... there were a couple of threads on exeactly 
this issue.

Very briefly

1. Beowulf clusters CANNOT help make Postgresql or any databases run 
faster. You need the database code to be modified to do that (think 
Oracle 10g). I met a company at Supercomputer 03 last year that had 
Mysql running on a cluster... you may wish to query for them.

2. You could try to sponsor the development of a parallel postrgresql - 
talk to the postgresql development team... when I broached the idea in 
1998.. there was some interest.. unfortunately.. I could not afford the 
development/sponsorship costs then.

3. Try running Postgresql on a cluster filesystem like PVFS - it is not 
gauranteed as it probably fails the ACID test for a SQL compliant 
database. The basic idea is that if we cannot parallelise the database - 
we make the underlying IO parallel and hence boost the IO performance of 
the system.. and any applications that run on them.. and this includes 
Postgresql.

Hope this helps.

Cheers!
Laurence
Scalable Systems
Singapore



Joshua Marsh wrote:
> Hi all,
> 
> I'm currently working on a project that will require fast access to
> data stored in a postgreSQL database server.  I've been told that a
> Beowulf cluster may help increase performance.  Since I'm not very
> familar with Beowulf clusters, I was hoping that you might have some
> advice or information on whether a cluster would increase performance
> for a PostgreSQL database.  The major tables accessed are around
> 150-200 million records.  On a stand alone server, it can take several
> minutes to perform a simple select query.
> 
> It seems like once we start pricing for servers with 16+ processors
> and 64+ GB of RAM, the prices sky rocket.  If I can acheive high
> performance with a cluster, using 15-20 dual processor machines, that
> would be great.
> 
> Thanks for any help you may have!
> 
> -Josh
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: laurenceliew.vcf
Type: text/x-vcard
Size: 150 bytes
Desc: not available
Url : http://www.scyld.com/pipermail/beowulf/attachments/20041027/bb931a9a/laurenceliew.vcf