<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div apple-content-edited="true"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br></div></span></div><div><blockquote type="cite"><div><font class="Apple-style-span" color="#000000"><br></font>Date: Mon, 11 Jan 2010 09:58:24 -0500<br>From: Michael Di Domenico <<a href="mailto:mdidomenico4@gmail.com">mdidomenico4@gmail.com</a>><br>Subject: Re: [Beowulf] cisco networking<br>To: Glen Dosey <<a href="mailto:doseyg@r-networks.net">doseyg@r-networks.net</a>><br>Cc: <a href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a><br>Message-ID:<br><span class="Apple-tab-span" style="white-space:pre"> </span><<a href="mailto:e75d22a91001110658k6b5e30eaq83e8a83080cc529@mail.gmail.com">e75d22a91001110658k6b5e30eaq83e8a83080cc529@mail.gmail.com</a>><br>Content-Type: text/plain; charset=ISO-8859-1<br><br>On Sun, Jan 10, 2010 at 1:29 AM, Glen Dosey <<a href="mailto:doseyg@r-networks.net">doseyg@r-networks.net</a>> wrote:<br><blockquote type="cite">You can have multiple ether-channel links between 2 switches. The<br></blockquote><blockquote type="cite">limitation is that if they are in the same layer 2 broadcast domain<br></blockquote><blockquote type="cite">(VLAN) only 1 will be active and STP will block the other.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">If you place each switch in a separate network you could use equal cost<br></blockquote><blockquote type="cite">load balancing across multiple point to point SVI's on ether-channel<br></blockquote><blockquote type="cite">links. You'll increase bandwidth at the cost of a little additional<br></blockquote><blockquote type="cite">latency. Of course at the bandwidths you are talking about (160 Gbit/s<br></blockquote><blockquote type="cite">or more ) I have no idea how it would really work. Backplane bandwidth,<br></blockquote><blockquote type="cite">hashing algorithms and all sorts of other factor come into play and<br></blockquote><blockquote type="cite">could cause it to fail completely. I'd love to try it out but I can't<br></blockquote><blockquote type="cite">help but think there is probably a better architectural solution.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">What are you trying to do ?<br></blockquote><br>The attempt is really to get a bunch of devices which only have<br>ethernet 1Gbps connectivity, to talk to a bunch of other machines with<br>Infiniband Connectivity. (Ie, compute nodes to storage). But we want<br>the aggregate bandwidth to be over 5GBps.<br><br>I was trying to see if there was a simple solution using some of the<br>existing stuff we have, without having to get overly creative with the<br>network or purchase 10G Ethernet to IB routers...<br><br>Looks like IB routers is the only solution that really works for me...<br><br><br>------------------------------<br><br></div></blockquote><div>Remember that you can actually use nodes to route IP between GbE and IB, just don't expect them to run at IB wirespeed. This can at least save you from paying a premium for proprietary hardware in order to test functionality. Last I tried 3-4 Gbit was about as fast as IP over IB would get through the Interface presumably due to IP overhead... but that was long ago so if you try it I'd love to hear your results.</div><div><br></div><div><br></div></div></body></html>