[Beowulf] DIMMnet-1 DIMM slot NIC?
joachim at ccrl-nece.de
Tue Jul 27 01:11:08 PDT 2004
Bill Broadley wrote:
> On Mon, Jul 26, 2004 at 06:26:59PM +0200, Joachim Worringen wrote:
>>Keith D. Underwood wrote:
>>>Now, if you want to implement something like UPC or SHMEM and you have
>>>all of the shared address space located on the card plugged into that
>>>slot (maybe not incredibly practical, but probably possible) then a DIMM
>>>interface may be your best bet.
>>Even then, you have the problem that this approach doesn't scale: a
>>single memory slot can cover only a small fraction of the memory of the
>>N other nodes (think of N >> 10), which requires to fall back on message
> What is your reasoning? Bandwidth limitations? Address bits? Latency?
As I wrote, not enough address space. A single DIMM can cover at most
50% of the address space of the local node, and never more than the
local node may have physiscally installed. But the accumulated memory of
all nodes in a cluster is much larger, and thus the fraction of remote
memory that a node could access directly through such a NIC would
decrease as the number of nodes in the cluster increases.
Thus, if you have 100 nodes, you can only access 1% of each nodes memory
directly. This may be sufficient for message passing, but not for
shared-memory programming models, or even RDMA.
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de
More information about the Beowulf