[Beowulf] Home beowulf - NIC latencies

[Patrick Geoffray]

Hi Vincent,

Vincent Diepeveen wrote:
>>>CPU's are 100% busy and after i know how many times a second the network
>>>can handle in theory requests i will do more probes per second to the
>>>hashtable. The more probes i can do the better for the game tree search.
>>
>>With a gigE network that sounds like 40us or so.  With Myrinet or IB
>>it's in the 4-6us range.  If you bought dual opterons with the special
> 
> 
> At the quadrics and dolphin homepage they both claim 12+ us for Myrinet.

Seriously, here are MPI latencies with MX on F cards on Opteron (PCI-X), 
that includes fibers and a switch in the middle:

    Length   Latency(us)    Bandwidth(MB/s)
         0       2.684          0.000
         1       2.874          0.336
         2       2.898          0.690
         4       2.978          1.343
         8       2.965          2.699
        16       2.993          5.347
        32       3.409          9.388
        64       3.563         17.960
       128       3.977         32.185
       256       5.699         44.916

Quadrics would be lower by a 1.5 us, I don't know about Dolphin, I 
didn't hear about noticeable SCI clusters in a long time.

> I am very impressed by the quadrics and dolphin cards. Probably by
> infinipath too when i check them out. Will do. 
> 
> I'm not so impressed yet by myrinet actually, but if cluster builders can
> earn a couple of hundreds of dollars more on each node i'm sure they'll do it.

I don't think Myrinet would be the cheapest, I am sure you can get a 
better deal from desperate interconnect vendors.

What does not impress you in Myrinet ?

Patrick
-- 

Patrick Geoffray
Myricom, Inc.
http://www.myri.com