[Beowulf] Q: IB message rate & large core counts (per node) ?
hahn at mcmaster.ca
Thu Feb 25 21:10:41 PST 2010
>> Regardless of how tight the seastar per-hop latency is,
>> IB has 2.5x the per-hop fanout (2or 3 outgoing 9.6 GB links
>> versus 18 outgoing 4 GB/s links). higher radix means an
>> advantage that increases with size.
> Doesn't this assume worst case all-to-all type communication
I'm assuming random point-to-point communication, actually.
> If you are just trading ghost cell data with your neighbors
> and you have placed your job smartly on the torus the fan out
> advantage mentioned is irrelevant. No?
if your comms are nearest-neighbor,
then yes, a nearest-neighbor fabric is your friend ;)
how often does that actually happen? to work out so neatly would
preclude, for instance, adaptive meshes, right? it seems like mostly
I see jobs with no obvious regular structure to their communication.
More information about the Beowulf