networking options

[Velocet]

On Tue, Sep 24, 2002 at 11:42:07AM -0500, Richard Walsh's all...
> 
> Matt Pratola wrote:
> 
> >Of course each app is different, but i'm interested more in high
> >throughput/low latency as it cannot harm apps which don't require this,
> >but is absolutely necessary for apps which do require this level of
> >performance.  In this regard, myrinet is certainly the one i've heard the
> >most about, but that doesn't preclude something else which is new and/or
> >better.
> 
> OK. Your most obvious choices in 2002 with rough performance estimates
> (these are generally better-than MPI numbers) are:
> 
>        Technology		~Best Case Bandwidth	~Hardware Latency
> 
>     1. Fast Ethernet*		   ~10 Mbytes/sec	~50-75  usecs
>     2. Channel Bonded FE	~20-25 Mbytes/sec	~50-75  usecs
>     3. Gigabit Ethernet*	~60-110 Mbytes/sec	~50-200 usecs

I remember reading here on beowulf much lower latencies than this for
Gig E - is it really as slow as FE (and with a higher upper limit?)
On the order of 5-10us  for good GBE nics, and 15-50us for good
switches. Does fibre or copper GBE make a diff?

What about for switchless GigE connections? (Useful in some cases where a ring
topology only is required - we used that in a cluser we built (with copper
Gige) as there was no internode communication except with adjacent nodes.
Saves alot of cash on an n-port GBE switch for a large cluser sized n. n
actually becomes much larger when you get rid of the switch but keep $
constant. :)

/kc


>     4. Channel Bonded GE	~????? Mbytes/sec	~50-200 usecs
>     5. SCI             		~150-200 Mbytes/sec	~4-8	usecs
>     6. Myrinet         		~200-250 Mbytes/sec	~7-10   usecs
>     7. Quadrics**      		~300 Mbytes/sec		~4-5    usecs
> 
>     8. Cray T3E 		~200-400 Mbytes/sec	~1-2	usecs
> 
>     *  GAMMA MPI can improve latencies signficantly.
>     ** Quadics claims 700 Mbytes/sec in 2003 (dual rail?)
> 
> Does not include less convention/available things like Infiniband or
> Fiber Channel. 
> 
> These data are from memory and subject to varification and refinement.
> You can check on proce-performance and scaling issues.
> 
> Hope this helps,
> 
> rbw
> #---------------------------------------------------
> # Richard Walsh
> # Project Manager, Cluster Computing, Computational
> #                  Chemistry and Finance
> # netASPx, Inc.
> # 1200 Washington Ave. So.
> # Minneapolis, MN 55415
> # VOX:    612-337-3467
> # FAX:    612-337-3400
> # EMAIL:  rbw at networkcs.com, richard.walsh at netaspx.com
> #         rbw at ahpcrc.org
> #
> #---------------------------------------------------
> # "What you can do, or dream you can, begin it;
> #  Boldness has genius, power, and magic in it."
> #                                  -Goethe
> #---------------------------------------------------
> # "Without mystery, there can be no authority."
> #                                  -Charles DeGaulle
> #---------------------------------------------------
> # "Why waste time learning when ignornace is
> #  instantaneous?"                 -Thomas Hobbes
> #---------------------------------------------------
> # "In the chaos of a river thrashing, all that water
> #  still has to stand in line."    -Dave Dobbyn
> #---------------------------------------------------
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Ken Chase, math at velocet.ca  *  Velocet Communications Inc.  *  Toronto, CANADA