custom cluster cabinets (was Re: 1U P4 Systems)

[Bari Ari]

Douglas Eadline wrote:

> Keep in mind there is no easy way to correlate dense CPUs or MFLOPS per
> watt to actual performance. Packing CPUs in a custom 1U box
> may be a big win for some problems, but a big waste of money
> for others.
> 
The StoneSoup BeoClusters will always be the best approach for many as 
others have pointed out here due to budget, reuse and 
management/organizational constraints.

> Also, the "lego we use" in the Beowulf community is largely
> what is produced for other much larger markets and while we would like
> to see some things done differently, there is a reasonable
> trade-off between cost/flexibility/performance.

The turn-key clusters of 16 nodes and greater targeted at high speed 
with high bandwith/low latency interconnects seems to be where things 
can be  improved significantly. Maybe some new "legos" are needed.

> 
> I find it interesting that in all the talk of P4 systems
> there is little discussion about the Intel chips-set
> for the P4 only supporting 32 PCI. If you need anything
> other than Fast Ethernet this could be a real drawback
> (i.e. an imbalanced system with bottle necks)
> The new Xeon chip set has 2 64-bit slots. Of course there
> are other chip-sets on the immediate horizon, but
> the market seems to be making a clear differentiation about the
> "desktop" vs. "server" product.

Some applications will churn away at a piece of data for hours or days 
before spitting out a few bits to pass on or compare to what the other 
nodes have as results. A 300bps link between nodes may be adequate here. 
Other applications may only run through a few CPU cycles before passing 
on a chunk of data where even 10Gb/sec interconnections are bottlenecks. 
It's this high speed end of clustering where I see a need for improvements.

I have seen interest in the P4 for it's raw performance only and not for 
the lack of current support for DDR and 64/66 PCI. It's all around the 
corner though from all the chipset vendors. Infinband will really make a 
big difference for high speed and high bandwidth applications when it 
comes standard in chipsets.

Bari