[Beowulf] Gigabit switch recommendations

[Yaroslav Halchenko]

Hi All Beowulfers,

Continuing reports Joshua was nice to share, I want to report some pilot
testing of the switch I got -- I went for a cheap (~700$ for 44 ports)
D-Link DGS-1248T. Did firmware upgrade on it as soon as it arrived (had
to use laptop with windows since firmware upgrade is available only in
the tool which shipped along -- web interface of the switch doesn't
allow it)

I banged by hand against the way for a few minutes since I could not
find any nice tool to perform parametric analysis/plotting of network
performance (please advice if there is any), thus I wrote two small
scripts - one to collect and another to plot the results. By default it
uses netperf and plots throughput between two nodes depending on MTU
sizes.

Scripts are available from
http://www.onerussian.com/Linux/scripts/netperf/test_netperf.sh
http://www.onerussian.com/Linux/scripts/netperf/plot_test.py

Graphs are available from
http://www.onerussian.com/Linux/scripts/netperf/examples/

Tuned parameters were (blindly copied from Joshua :-))
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_rmem = 4096 16777216 16777216
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608

whenever defaults are
net.ipv4.tcp_wmem = 4096        16384   131072
net.ipv4.tcp_rmem = 4096        87380   174760
net.core.wmem_max = 131071
net.core.rmem_max = 131071

Results:

 weirdo -- needs to rerun may be -- there is a range of MTUs (0 - ~4500)
 when switch performed better than crossover with the same network parameters 

 MTUs 3500-4000 seems to be the best in terms of throughput (~185000 KBps
 duplex) and tuning the params seems to help in general but for
 3500 seems to be not that important (need to run with smaller step of
 MTU)

 going away from default MTU (1024) helps a lot for CPU utilization, so
 my NFS server should be happier than now. but "tuned" params incure
 their own impact on cpu utilization, thus they also has to be checked
 for their optimal values for the task

 generally this DLink seems to be not as good as Joshua's SMC on high
 MTU sizes -- speed between crosslinked and via-switch diverges
 considerably (around 30000-35000 KBps loss)

I hope this is of help for anyone ;)

-- 
                                  .-.
=------------------------------   /v\  ----------------------------=
Keep in touch                    // \\     (yoh@|www.)onerussian.com
Yaroslav Halchenko              /(   )\               ICQ#: 60653192
                   Linux User    ^^-^^    [175555]


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20060421/5f9b2518/attachment.sig>