[Beowulf] How to configure a cluster network

[Daniel Pfenniger]

Andrew,

Here are joined some possible topologies I was contemplating, with some
remarks about them.  Many other topologies are possible.

The first one is the one you mention.  If 12 nodes linked to one switch 
communicate with 12 nodes on another switch the bandwidth  is reduced to
8/12 = 2/3.  All the packets needs either 1 or 3 hops through a switch.

The second topology improves the bandwidth between the core and edge
switches.  The bandwidth for the above case is not reduced except that 
some routes need 4 hops.

The third topology has the feature that 2 hops node to node 
communications are possible, but global communications are slightly 
degraded with respect to the previous case.

In the fourth case we have one core switch and 4 edge switches. When
10 nodes communicate with 10 other nodes on another edge switch
2 or 3 routes need 2 hops and the rest 3 hops, without bandwidth 
reduction.  It seems to me that this topology is better than the 
previous ones.

Finally the last topology has no core switch.  All the routes need 
either 1 or 2 hops.  This one seems to me even better.

Since I am not network expert I would be glad if somebody explains
why the first solution is the best one.

	Dan



andrew holway wrote:
> Daniel
> 
> To give a half bisectional bandwidth the best approach is to set up
> two as core switches and the other 4 as edge switches.
> 
> Each edge switch will have four connections to each core switch
> leaving 16 node connections on each edge switch.
> 
> Should provide a 64 port network.
> Make sense?
> 
> Ta
> 
> Andy
> 
> 
> On Thu, Jul 24, 2008 at 11:06 AM, Daniel Pfenniger
> <Daniel.Pfenniger at obs.unige.ch> wrote:
>> Hi,
>>
>> I have the problem of connecting with InfiniBand 50 1-HCA nodes with 6
>> 24-port switches.  Several configurations may be imagined, but which one is
>> the best?  What is the general method to solve such a problem?
>>
>> Thanks,
>>
>>        Dan
>>
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Connexion of 50 nodes with 6 24-port IB switches.pdf
Type: application/pdf
Size: 12626 bytes
Desc: not available
Url : http://www.scyld.com/pipermail/beowulf/attachments/20080724/4a55ab58/Connexionof50nodeswith624-portIBswitches.pdf