[Beowulf] AMD’s Direct Connect Architecture

[Richard Walsh]

On 30 Jun 2004 16:23:50 -0600 Keith Underwood wrote:

>On Wed, 2004-06-30 at 14:52, rbw at ahpcrc.org wrote:
>> On Mon, 28 Jun 2004 11:52:34 -0700, Greg Lindahl wrote:
>> 
>> >Red Storm is 1-way nodes.
>> 
>> Right, but there is room for confusion here because the
>> node boards include 4 Opteron processors ... yes, each 
>> with their own directly controlled memory ... but intra-
>> board accesses will be on a short hyper transport path 
>> through the local router.
>
>On Red Storm (as it is being delivered to Sandia), there is no practical
>difference between communications on a board and communications off of a
>board.  Both go through the NIC/router.  Ok, so going off board could
>cost you a few extra nanoseconds of wire delay, but.... 
>
> Keith

Yes, yes ....  I was trying to point out the potential  
for confusion in looking just at the board and created by 
the fact that SMP-like sharing of hardware is blurring as 
designs share more or less of the components on a board, 
module, chip ... so we have dual core chips and multi-chip
modules sharing cache on the same board that may or may
not share the bus to memory or memory directly.  Siamese
twining of hardware can take many forms and what's on the
same board may mean something or not. In this case it means
only a few microseconds reference latency advantage.

Is a dual Opteron motherboard with a memory channel from
each processor to its own memory and also hypertransport
memory sharing a 2-way system or just 2 1-way systems
on the same board? Perhaps 1.333 1.5-way systems? ;-)

rbw