[Beowulf] EM64T Clusters

[Patrick Geoffray]

Mark Hahn wrote:
> can anyone offer hints of pci-express performance?  for instance,
> boards like:
> 
> http://www.tyan.com/products/html/tomcati915.html
> 
> have a x16 PCI-e slot, which afaikt, would be a perfect place to put
> a low-latency cluster interconnect board.  I haven't heard Quadrics
> or Myri talking about their roadmap for PCI-e, but IB people seem to 

Myrinet 10G uses the 10 Gigabit Ethernet PHY (12.5 Gb/s signal rate and 
10 Gb/s data rate, to not be confused with 10 Gb/s signal and 8 Gb/s 
data rate). XAUI protocol. PCI-E interface, following by a HT version.
4Q04.

The IO bus counts for 30% of the 3.5us MPI latency on E cards with 
Opteron and its very good HT-PCIX bridge. PCI-E will divide the IO bus 
overhead roughly by 2, but other side effects of the PCI Express design 
like the implicit doorbell on PIO will also save cycles in the critical 
path. I would say latency will go down to 2.5us, but it will depend of 
the PCI-Express host implementation.

Patrick
-- 

Patrick Geoffray
Myricom, Inc.
http://www.myri.com