[Beowulf] DIMMnet-1 DIMM slot NIC?

[Joachim Worringen]

Bill Broadley wrote:
> On Mon, Jul 26, 2004 at 06:26:59PM +0200, Joachim Worringen wrote:
> 
>>Keith D. Underwood wrote:
>>
>>>Now, if you want to implement something like UPC or SHMEM and you have
>>>all of the shared address space located on the card plugged into that
>>>slot (maybe not incredibly practical, but probably possible) then a DIMM
>>>interface may be your best bet.
>>
>>Even then, you have the problem that this approach doesn't scale: a 
>>single memory slot can cover only a small fraction of the memory of the 
>>N other nodes (think of N >> 10), which requires to fall back on message 
> 
> What is your reasoning?  Bandwidth limitations?  Address bits?  Latency?

As I wrote, not enough address space. A single DIMM can cover at most 
50% of the address space of the local node, and never more than the 
local node may have physiscally installed. But the accumulated memory of 
all  nodes in a cluster is much larger, and thus the fraction of remote 
memory that a node could access directly through such a NIC would 
decrease as the number of nodes in the cluster increases.

Thus, if you have 100 nodes, you can only access 1% of each nodes memory 
directly. This may be sufficient for message passing, but not for 
shared-memory programming models, or even RDMA.

  Joachim

-- 
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de