[Beowulf] Gbe switch comparision

[Mikhail Kuzminsky]

In message from H?kon Bugge <Hakon.Bugge at scali.com> (Thu, 30 Sep 2004 
10:33:48 +0200):
>Hi,
>
>we did a study of two different Gigabit ethernet switches from one 
>particular vendor on a 16-node XEON cluster (2.4GHz dual XEON, 400MHz 
>FSB, Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet (rev 
>15)). We run a number of benchmarks every night on the cluster as 
>part of our regression testing for Scali MPI Connect(synthetic, 
>application benchmarks and applications). Although we ran on a _very_ 
>little system, that is not of the fastest these days, we observed 
>significant differences. I thought the findings might be of interest 
>to the community, not necessarily as a guide to which switch to 
>purchase, but merely as a motivation to assess the quality of the 
>switches to consider in a purchase.

   This data are very interesting. Sorry, what is known about jumbo 
frames support by this switches ? BTW, is somewhere available some 
comparison of TCP throughput (for example, TCP_STREAM from netperf) 
w/jumbo frames enabled vs disabled - for any NICs/switches ?

Yours
Mikhail Kuzminsky
Zelisnky Institute of Organic Chemistry
Moscow


>
>Best regards, Hakon
>
>
>Benchmark                Switch A   Switch B       A/B
>======================================================
>Pnt2pnt (usec - lower is better)
>Ping pong latency            30.9       34.6
>Ping pong bw 128              3.6        3.3
>Ping pong bw 1k              16.8       14.2
>Ping pong bw 1M             116.6      116.4
>All2all 1k                  57.57       60.4
>All2all 128k                78.93      79.57
>Barrier latency            139.27      143.9
>-------------------------------------------------------
>Pallas snapshots (usec - lower is better)
>Reduce 4                   250.79     268.91      0.933
>Redyce 1M                   61241      34575      1.771
>Allreduce 4                167.69     172.39      0.973
>Allreduce 1M                62540      41208      1.518
>Bcast 4                    255.23     250.43      1.019
>Bcast 1M                    72173      72227      0.999
>Allgather 4                571.33     634.86      0.900
>Allgather 1M               612035     617526      0.991
>Alltoall 4                 255.27     254.77      1.002
>Alltoall 1M               1500884     856977      1.751
>-------------------------------------------------------
>NPB (Total Mflops/sec - higher is better)
>bt                         3631.4     4800.8      0.756
>cg                         313.15    1628.62      0.192
>ep                         387.49     389.58      0.995
>ft                         2643.3     3985.6      0.663
>is                          61.29     168.56      0.364
>lu                         8835.2    10306.5      0.857
>mg                         1760.9     2913.8      0.604
>sp                         1499.4     2377.4      0.631
>-------------------------------------------------------
>Applications (Jobs/day - higher is better)
>MM5                          4.64       4.84      0.958
>Fluent fl5m3                475.8      499.3      0.953
>Fluent fl5l2               1122.1     1116.3      1.005
>Eclipse E100 100k          178.14     188.24      0.946
>Eclipse E100 1M             28.13      29.54      0.952
>Eclipse E300 100k           84.74     105.45      0.804
>Ls-dyna                      44.7      44.49      1.005
>Powerflow                  413.48     414.69      0.997
>StarCD Aclass              141.34     144.18      0.980
>StarCD Eng20              1367.09    1353.81      1.010
>StarCD large custom         40.13      39.76      1.009
>
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit 
>http://www.beowulf.org/mailman/listinfo/beowulf