[Beowulf] 1.2 us IB latency?

[Peter Kjellstrom]

On Wednesday 28 March 2007, Mark Hahn wrote:
> >> start timer
> >> send(other,small-message)		recv(first,small-message)
> >> recv(other,small-message)		send(first,small-message)
> >> stop timer
> >>
> >> I'll actually see 2.4 us between the timer calls?  if I understand,
> >> aggregation would only help on a streaming test.  in fact, this kind
> >> of isolated RPC-like exchange is what I see most commonly.
> >
> > Assuming you could time it with any accuracy, yes.
>
> that's not an issue - rdtsc is perfectly good into the tens of ns range.

I'll have to hack together a rdtsc based mpi microbenchmark some day it seems 
=)

> > I've seen ib_write_lat figures of ~1.
>
> well, I was assuming mpi - does anyone really write apps using ib
> primitives?

Sorry for being unclear here. What I wanted to say was that, unrelated to 1.5 
us ping-pong on mpi I have also observed verbs level latency (ib_write_lat) 
of around 1 us. And that figure is not affected by any mvapich trickery :-).

/Peter

> in anycase, if this is a non-streaming latency result, it's pretty good;
> enough to make quadrics look comprehensively out of the picture (guess
> they've switched horses to 10GE anyway.)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.scyld.com/pipermail/beowulf/attachments/20070328/3d45629e/attachment.bin