No subject


Tue Nov 9 01:00:01 PST 2010


should be using as much of one language as possible.  mod_perl would be my
choice - you'll have more flexibility with mod_perl because Apache is more
accessable overall to developers, IMHO.

> 
> The database question. The database is large, but entirely static.
> Would PostgreSQL be able to handle this comparably to Oracle? The 
> licenses for Oracle are not so very expensive anymore, but if we're
> going towards a farm of boxes, this will start to cumulate. Otoh,
> some Oracle stuff seems to be able to handle parallel databases
> natively. Hmm.

I think it might!  I'm working with PostgreSQL, and I'm seeing comparable
performance, but you'll need to do some benchmarking.  If it doesn't compare
now, it eventually will.  Watch out for using joins in PostgreSQL - they can
impact run times.

> 
> Sorry for the bunch of clueless questions, but I'm not exactly a
> computer person, nor is the company extremely competent in technical
> questions (they're all a bunch of chemists, which just have been
> working with computers for a long time). The budget for hardware seems
> to be there, but I'm a new guy, and I can't afford buying a bunch
> of expensive crap which will just sit there gathering dust.

You're not clueless 'cause you're asking the right questions - this is the
same type of thing directors and managers must contend with.  You have a
chance to sell beowulf to your higher-ups.  Go for it!



> 
> I would also welcome some pointers towards lists where questions
> such as these are handled holistically. I'd settle for a bunch
> of dedicated lists too, though.


> 
> <off-topic/>
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) 
> visit http://www.beowulf.org/mailman/listinfo/beowulf
> 

------_=_NextPart_001_01C092C6.98BBCDA0
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2650.12">
<TITLE>RE: phasing out Solaris/Oracle/Netscape with =
Linux/PostgreSQL/Apache</TITLE>
</HEAD>
<BODY>

<P><FONT SIZE=3D2>Let me know if I can help beyond what I write =
below.&nbsp; Additional comments from the crowd welcome.&nbsp; See =
below . . . </FONT>
</P>

<P><FONT SIZE=3D2>&gt; -----Original Message-----</FONT>
<BR><FONT SIZE=3D2>&gt; From: Eugene.Leitl at lrz.uni-muenchen.de</FONT>
<BR><FONT SIZE=3D2>&gt; [<A =
HREF=3D"mailto:Eugene.Leitl at lrz.uni-muenchen.de">mailto:Eugene.Leitl at lrz=
.uni-muenchen.de</A>]</FONT>
<BR><FONT SIZE=3D2>&gt; Sent: Thursday, February 08, 2001 10:43 =
PM</FONT>
<BR><FONT SIZE=3D2>&gt; To: linux-elitists at zgp.org; =
pigdog-l at bearfountain.com;</FONT>
<BR><FONT SIZE=3D2>&gt; beowulf at beowulf.org</FONT>
<BR><FONT SIZE=3D2>&gt; Subject: phasing out Solaris/Oracle/Netscape =
with</FONT>
<BR><FONT SIZE=3D2>&gt; Linux/PostgreSQL/Apache</FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; This is somewhat off-topic, so bear with =
me.</FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; Um, I need some advice. The company I'm with =
has a Solaris Sun with</FONT>
<BR><FONT SIZE=3D2>&gt; Netscape Enterprise Server. It is running a mix =
of C, Perl, Oracle</FONT>
<BR><FONT SIZE=3D2>&gt; and Daylight. The latter beast currently =
requires 1.5 GByte RAM,</FONT>
<BR><FONT SIZE=3D2>&gt; but is still creaking at all seams. We're going =
to kick Daylight </FONT>
<BR><FONT SIZE=3D2>&gt; sooner or later, so the memory footprint will =
drop, possibly a lot.</FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; I don't yet understand the architecture of the =
chemical database</FONT>
<BR><FONT SIZE=3D2>&gt; we're running, so I don't know where the =
bottlenecks are, but I need</FONT>
<BR><FONT SIZE=3D2>&gt; to build a Linux machine which could outperform =
the Sun at a fraction</FONT>
<BR><FONT SIZE=3D2>&gt; of the price. I will set it up locally, and =
hammer it with queries,</FONT>
<BR><FONT SIZE=3D2>&gt; doing measurements, stability tests, =
etc.</FONT>
</P>

<P><FONT SIZE=3D2>This could get you.&nbsp; Critical to understand some =
basic things about the existing application.&nbsp; Exporting data, etc =
is your first concern - you need to know how to do that.&nbsp; That =
could very well set the pace of a conversion.</FONT></P>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; I would rather like to use Athlons, especially =
recent DDR </FONT>
<BR><FONT SIZE=3D2>&gt; Dual Athlons,</FONT>
<BR><FONT SIZE=3D2>&gt; but this doesn't seem to go too well with =
required stability. This</FONT>
<BR><FONT SIZE=3D2>&gt; means I should use a dual Pentium III =
motherboard, right? Does Linux</FONT>
<BR><FONT SIZE=3D2>&gt; handle these well? I heard these can take up to =
8 GByte RAM, I think </FONT>
<BR><FONT SIZE=3D2>&gt; I should start with 2 GBytes. I don't think the =
application is CPU </FONT>
<BR><FONT SIZE=3D2>&gt; bound, but I don't know for sure yet. It =
certainly seems to exercise </FONT>
<BR><FONT SIZE=3D2>&gt; disks strongly, so here is another =
question:</FONT>
</P>

<P><FONT SIZE=3D2>Linux does handle them well, but I'm impartial to =
FreeBSD.&nbsp; Since you're using Sun OS already, you could continue =
with Sun X86 version without making the big switch all at once - it's =
free and a works well enough.&nbsp; It would make a fine part of a =
beowulf, and allow you to get used to Linux over a longer period of =
time.&nbsp; If you can make the transition in baby steps, it's =
better.</FONT></P>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; do I absolutely, positively need SCSI? I was =
thinking about putting a</FONT>
<BR><FONT SIZE=3D2>&gt; second 100 EIDE host adapter in, and run disk =
striping plus mirroring</FONT>
<BR><FONT SIZE=3D2>&gt; over 4 EIDE hard drives (the better models from =
IBM). Or should I</FONT>
<BR><FONT SIZE=3D2>&gt; use a Dual-Pentium mumboard with onboard SCSI, =
and buy several fast,</FONT>
<BR><FONT SIZE=3D2>&gt; hot &amp; noisy scuzzys, soft-RAIDing them? =
Perhaps even harware RAID?</FONT>
<BR><FONT SIZE=3D2>&gt; I don't think the disks need to be very large, =
but they </FONT>
<BR><FONT SIZE=3D2>&gt; better be fast.</FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
</P>

<P><FONT SIZE=3D2>SCSI is GREAT, and you should set up redundant hot =
swaps so if you crash, you insert a new disk, type &quot;boot&quot;, =
and you're back online with a node.&nbsp; I think Sun stations =
outperform the Intel boards on disk throughput, but you could =
check.</FONT></P>

<P><FONT SIZE=3D2>&gt; I would like to use Reiser FS, so this only =
allows me RAID 0/1, right?</FONT>
<BR><FONT SIZE=3D2>&gt; Higher stability, lack of fscking delay in case =
machine needs to be</FONT>
<BR><FONT SIZE=3D2>&gt; rebooted and no 2 GByte file size limit would =
seem to be needed.</FONT>
<BR><FONT SIZE=3D2>&gt; Since this is ReiserFS, I should obviously go =
with the latest stable</FONT>
<BR><FONT SIZE=3D2>&gt; kernel. I've been using Mandrake for my desktop =
and small </FONT>
<BR><FONT SIZE=3D2>&gt; time testing, </FONT>
<BR><FONT SIZE=3D2>&gt; but for this application this is probably not =
the way to go. Which </FONT>
<BR><FONT SIZE=3D2>&gt; distro should I choose? Debian?</FONT>
</P>

<P><FONT SIZE=3D2>Not sure on this one.&nbsp; Not enough exposure with =
all of those filesystems.&nbsp; I know, use MFS, and load up machines =
with 2GB of ram!&nbsp; Just kidding.</FONT></P>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; We seem to be moving towards an architecture =
consisting of a bunch </FONT>
<BR><FONT SIZE=3D2>&gt; of Perl programs (it's not settled, but the few =
routines we have are</FONT>
<BR><FONT SIZE=3D2>&gt; in Perl) communicating via sockets. (Right now =
it's a mix of C and </FONT>
<BR><FONT SIZE=3D2>&gt; Perl, talking via named pipes). The queries (a =
chemical structure </FONT>
<BR><FONT SIZE=3D2>&gt; drawn within a browser, using a Java applet or =
a plug-in) are </FONT>
<BR><FONT SIZE=3D2>&gt; handled with a dispatcher (a cgi-bin Perl =
thing). Sooner or </FONT>
<BR><FONT SIZE=3D2>&gt; later we will spread the query to a farm of =
boxes, each containing </FONT>
<BR><FONT SIZE=3D2>&gt; a database, or segmenting a database across =
several, cheap, redundant </FONT>
<BR><FONT SIZE=3D2>&gt; boxes. (But where not there yet). There's no =
alternative to sockets, </FONT>
<BR><FONT SIZE=3D2>&gt; right?</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>You'll take a big performace hit running perl too =
much - it's interpreted.&nbsp; I know it's popular but with a big =
project, you should get the training required to do it in a compiled =
language, like C/C++.&nbsp; If you insist on an interpreted language, =
use Java - it's a 'cleaner' language to manage on a large scale.&nbsp; =
For socket work I would definitely stick with C - and create a library =
specific to your needs that your perl programs could call.</FONT></P>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; Right now a query can take up to minutes, so I =
don't think mod_perl</FONT>
<BR><FONT SIZE=3D2>&gt; is needed. We don't get a lot of query hits, at =
least not yet. Should </FONT>
<BR><FONT SIZE=3D2>&gt; I try using Apache mod_perl instead of the =
Netscape Server </FONT>
<BR><FONT SIZE=3D2>&gt; nevertheless? </FONT>
</P>

<P><FONT SIZE=3D2>How much data does the longer queries access?&nbsp; =
If you're seeing less than sub-minute response times, it's a red =
flag.&nbsp; That's just a rule of&nbsp; thumb I use.&nbsp; I've seen =
five second return times on several gigs of data with a Sun and =
Oracle.&nbsp; </FONT></P>

<P><FONT SIZE=3D2>From the standpoint of managing the code, regardless =
of what you use, you should be using as much of one language as =
possible.&nbsp; mod_perl would be my choice - you'll have more =
flexibility with mod_perl because Apache is more accessable overall to =
developers, IMHO.</FONT></P>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; The database question. The database is large, =
but entirely static.</FONT>
<BR><FONT SIZE=3D2>&gt; Would PostgreSQL be able to handle this =
comparably to Oracle? The </FONT>
<BR><FONT SIZE=3D2>&gt; licenses for Oracle are not so very expensive =
anymore, but if we're</FONT>
<BR><FONT SIZE=3D2>&gt; going towards a farm of boxes, this will start =
to cumulate. Otoh,</FONT>
<BR><FONT SIZE=3D2>&gt; some Oracle stuff seems to be able to handle =
parallel databases</FONT>
<BR><FONT SIZE=3D2>&gt; natively. Hmm.</FONT>
</P>

<P><FONT SIZE=3D2>I think it might!&nbsp; I'm working with PostgreSQL, =
and I'm seeing comparable performance, but you'll need to do some =
benchmarking.&nbsp; If it doesn't compare now, it eventually =
will.&nbsp; Watch out for using joins in PostgreSQL - they can impact =
run times.</FONT></P>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; Sorry for the bunch of clueless questions, but =
I'm not exactly a</FONT>
<BR><FONT SIZE=3D2>&gt; computer person, nor is the company extremely =
competent in technical</FONT>
<BR><FONT SIZE=3D2>&gt; questions (they're all a bunch of chemists, =
which just have been</FONT>
<BR><FONT SIZE=3D2>&gt; working with computers for a long time). The =
budget for hardware seems</FONT>
<BR><FONT SIZE=3D2>&gt; to be there, but I'm a new guy, and I can't =
afford buying a bunch</FONT>
<BR><FONT SIZE=3D2>&gt; of expensive crap which will just sit there =
gathering dust.</FONT>
</P>

<P><FONT SIZE=3D2>You're not clueless 'cause you're asking the right =
questions - this is the same type of thing directors and managers must =
contend with.&nbsp; You have a chance to sell beowulf to your =
higher-ups.&nbsp; Go for it!</FONT></P>
<BR>
<BR>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; I would also welcome some pointers towards =
lists where questions</FONT>
<BR><FONT SIZE=3D2>&gt; such as these are handled holistically. I'd =
settle for a bunch</FONT>
<BR><FONT SIZE=3D2>&gt; of dedicated lists too, though.</FONT>
</P>
<BR>

<P><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; &lt;off-topic/&gt;</FONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
<BR><FONT SIZE=3D2>&gt; =
_______________________________________________</FONT>
<BR><FONT SIZE=3D2>&gt; Beowulf mailing list, =
Beowulf at beowulf.org</FONT>
<BR><FONT SIZE=3D2>&gt; To change your subscription (digest mode or =
unsubscribe) </FONT>
<BR><FONT SIZE=3D2>&gt; visit <A =
HREF=3D"http://www.beowulf.org/mailman/listinfo/beowulf" =
TARGET=3D"_blank">http://www.beowulf.org/mailman/listinfo/beowulf</A></F=
ONT>
<BR><FONT SIZE=3D2>&gt; </FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C092C6.98BBCDA0--




More information about the Beowulf mailing list