[Beowulf] 2.6.11 is out; with InfiBand support

[Don Holmgren]

On Thu, 3 Mar 2005, Greg Lindahl wrote:

> On Thu, Mar 03, 2005 at 08:58:33AM -0600, Don Holmgren wrote:
>
> > I was just quoted quantity 2 PCI-X HCA's (4x 2 port Mellanox) by a
> > reseller for $655 each.  Last Fall we purchased a large quantity of
> > PCI-E HCA's for considerably less than that unit price.
> >
> > Supposedly the "memory free" PCI-E HCA's that use host memory, rather
> > than on-board sram, should move prices towards $100 sometime this
> > year - we'll see (landed on motherboard pricing at ~ $70, see
> >    http://www.mellanox.com/news/press/pr_030105.html)
>
> You're mixing retail price with wholesale price. The $69 price is
> apparently quantity 10,000, for just the chip, and the card that
> is inexpensive will be PCIe 4X, which hurts performance.

Yes, it will be interesting to see what the motherboard vendors charge
for an IB option.  $69 (assuming they hit the 10K volume) + the price of
the I/O connector + engineering cost + margin.  I'm hoping that it will
be < $150, but I may be too optimistic.

Interesting comment about PCIe 4X hurting performance, thanks!  The
current PCI-E cards have two ports and use an 8X slot, but I'd guess
that most cluster applications use only a single port.  What's the
performance penalty for using a single 4X port HCA in a 4X PCI-E slot
compared with using a single port on a dual port card in an 8X slot?

I believe the MemFree cards also incur a few tenths of a microsecond
latency hit because of the need to access host memory, at least
accordinòg to the preliminary benchmarks shown at SC'04.


>
> I hear that today's street price for Mellanox-based cards is ~$600 in
> cluster-sized quantities, which matches what you report.

We did a little better than that - for quantity 260, we paid < $450 for
PCI-E HCA's.  A couple of other bids were around $500.


>
> > Like other high performance network gear, it's tough to get
> > accurate pricing information without going out and getting quotes.
>
> One nice thing about Myricom is that their prices have always been on
> the web -- all you need to know in addition is your discount.
>
> -- greg


Agreed.  Among the many other nice things about Myricom.

Don