[Fwd: Re: 32-port gigabit switch]

[Josip Loncaric]

Mark Hahn wrote:
>>>>clusters.  Remember on clusters latency is more important than
>>>>throughput. 
>>>
>>>Bzzt. Wrong. Dead Wrong. "It depends" is the correct answer. (Don't
>>>pass GO, don't collect $200).
> 
> 
> it does depend.  
> but as generalizations go, bandwidth is easy; latency is hard.

Agreed.  The "it depends" part can be seen in tests done at Dell:

http://ftp.us.dell.com/app/3q01-hpc.pdf

The second paper

http://ftp.us.dell.com/app/4q02-Riz.pdf

shows that on Gaussian elimination linear algebra problems "size-major" 
process mapping on clusters of dual CPU nodes connected via Fast Ethernet 
works better than "frequency-major" process mapping.  Faster networks (Gigabit 
Ethernet, Myrinet) reduced the advantage of "size-major" process mapping but 
did not completely eliminate it.  In other words, their Linpack test benefits 
more from increased bandwidth than from decreased latency. This is just one 
example; latency sensitive codes may lead to the opposite conclusion.

Sincerely,
Josip

P.S. My own experience with NPB is similar to the first Dell paper:
http://research.nianet.org/~josip/MPIonCoral.html

-- 
Dr. Josip Loncaric, Research Fellow              mailto:josip at nianet.org
National Institute of Aerospace       http://research.nianet.org/~josip/
144 Research Drive                       mailto:j.loncaric at larc.nasa.gov
Hampton, VA 23666, USA         Tel. +1-757-766-1395  Fax +1-757-766-1401