very high bandwidth, low latency manner?

[Steffen Persvold]

On Fri, 12 Apr 2002, Craig Tierney wrote:

> On Fri, Apr 12, 2002 at 01:40:08AM -0400, Patrick Geoffray wrote:
> > Steffen Persvold wrote:
> >
> > >>I talked to a guy at SC2002 from Quadrics and he said
> > >>that list pricing on a Quadrics network was about $3500
> > >>per node when you are in the 100s of nodes and up.
> > >>The price includes the cards, cables, switches,
> > >>etc.  This doesn't include any sort of discount that you
> > >>might get.  Myrinet is about $2000 for an equivelent
> > >>network at list price.   Dolphin/SCI falls around $2245 list
> > >>per node (if the system is > 144 nodes and you have to get
> > >>the 3d card).
> >  >
> > > This is list prices for the cards only, right ?
> >
> > Not for Myrinet. Actually $2000 per node is the total cost
> > (NIC/cable/port/software) for the high-end products (with L9/200 MHz),
> > should be more like $1500 for low-end ones. Craig is spoiled, only buys
> > the top stuff :-)
>
> Sorry Patrick.  The problem with trying to state numbers is that
> if you get it wrong, then the real knowledgeable ones can point it out.
>
> I figure out list cost on a 256 node system at about $2000 before for
> basic hardware.  I as wrong.  I reworked it and it is $1500 for
> 256 (and would be the same for 512 and 1024).

So what is wron with my calculations :

256 node L9/2MB/133MHz config :

M3F-PCI64B-2 NICs           256 *  $1,195 = $305,920
M3-E128 Switch enclosures     6 * $12,800 =  $76,800
M3-SW16-8F "Leaf" cards      32 *  $2,400 =  $76,800
M3-SPINE-8F "Spine" cards    64 *  $1,600 = $102,400
-----------------------------------------------------
Total cost                                = $561,920
Node cost                                 =   $2,195

and for a L9/2MB/200MHz config :

M3F-PCI64B-2 NICs           256 *  $1,495 = $382,720
M3-E128 Switch enclosures     6 * $12,800 =  $76,800
M3-SW16-8F "Leaf" cards      32 *  $2,400 =  $76,800
M3-SPINE-8F "Spine" cards    64 *  $1,600 = $102,400
-----------------------------------------------------
Total cost                                = $638,720
Node cost                                 =   $2,495

And this is without cable cost (since I don't quite know the cable
requirements for the total system, but atleast it is approx $100 per
node).

>
> I thought I had decent information.  What I was trying to provide was
> info on the three options.  The SCI price is per node (cards, cables,
> software).  It is $2245 list.  This would be appropriate for systems over
> 144 nodes.
>

Actually, Wulfkit3 comes in two flavors wether you want 1U or can manage
with a 2U (or higher) solutuion. The 1U is $2,445 and the 2U solution is
$2,245 per node.

> The Quadrics number I got from the rep might have been the sales number,
> and not a perfect comparsion to all the required hardware for a system of
> a few hundred nodes.
>

Now we have price comparisons for the interconnects (SCI,Myrinet and
Quadrics). What about performance ? Does anyone have NAS/PMB numbers for
~144 node Myrinet/Quadrics clusters (I can provide some numbers from a 132
node Athlon 760MP based SCI cluster, and I guess also a 81 node PIII ServerWorks
HE-SL based cluster).

Regards,
--
  Steffen Persvold   | Scalable Linux Systems |   Try out the world's best
 mailto:sp at scali.com |  http://www.scali.com  | performing MPI implementation:
Tel: (+47) 2262 8950 |   Olaf Helsets vei 6   |      - ScaMPI 1.13.8 -
Fax: (+47) 2262 8951 |   N0621 Oslo, NORWAY   | >320MBytes/s and <4uS latency