networking options

Richard Walsh rbw at
Wed Sep 25 12:55:11 PDT 2002

Joachim Worringen wrote:

>Do you have an example for this? AFAIK, the hop latency for each SCI node
>in-between is a few ns, and with a 128 node cluster in 3D topology, you would
>never pass more than about 15 nodes. Numbers from a real, busy cluster would
>be helpful.

 Yes very much ...

 But I do not ...  and now think I spoke prematurely based on the logical 
 idea that hopping costs would be subtantial. As you have seen, others
 have data from relatively quiet networks/rings of 8 nodes and have 
 extrapolations to 8x8x8 systems that show minimal latency growth. So
 unlike available bandwidth per node, SCI latency appears relatively
 stable as the number of nodes per ring grows under these test loads.

 That latency should grow in a high traffic environment is logical,
 but as I have proven once already "logic is the art of going wrong
 with confidence" ... so I will defer to those with data if there
 are any.

 Glad to see the product experts chiming in with better numbers and
 hope my interconnect table decreased rather than increased the amount 
 of entropy on the subject of interconnects here.



More information about the Beowulf mailing list