[Beowulf] 512 nodes Myrinet cluster Challanges

[Bruce Allen]

I can (somewhat) answer this question in the case of Supermicro H8SSL-i 
motherboards with the Supermicro plug in IPMI2 card.  The situation is 
identical to what you describe: the motherboard has two copper gigE ports, 
and the IPMI card 'piggybacks' on one of those ports.

The anwer is 'it depends upon your channel bonding implementation'.  If 
you use IEEE 802.3ad LACP bonding, then the data path between any two 
hosts will be on exactly one of the two link aggregated network lines. 
The algorithm which determines WHICH line carries the data between those 
hosts is determined by the sender and is not specified by the standard. 
It is typically a hash of the MAC addresses of sender and receiver, mod N. 
Sometimes it is just the XOR of the bottom few bits of sender and receiver 
MAC, mod N.  Some switches (Force10) let you choose an algorithm.  So does 
the linux channel bonding driver.

IF the link aggregation algorithm sends data over the line which IS 
connected to the IPMI card, then you CAN talk to it.  IF the algorithm 
sends data over the line which is NOT connected to the IPMI card, then you 
can't talk with it.

Bottom line: it's possible but will require time, testing, and 
experimentation on your part.

Cheers,
 	Bruce


On Fri, 5 May 2006, Alan Louis Scheinine wrote:

> Since you'all are talking about IPMI, I have a question.
> The newer Tyan boards have a plug-in IPMI 2.0 that uses
> one of the two Gigabit Ethernet channels for the Ethernet
> connection to IPMI.  If I use channel bonding (trunking) of the
> two GbE channels, can I still communicate with IPMI on Ethernet?
> best regards,
> Alan Scheinine
> -- 
>
> Centro di Ricerca, Sviluppo e Studi Superiori in Sardegna
> Center for Advanced Studies, Research, and Development in Sardinia
>
> Postal Address:               |  Physical Address for FedEx, UPS, DHL:
> ---------------               |  -------------------------------------
> Alan Scheinine                |  Alan Scheinine
> c/o CRS4                      |  c/o CRS4
> C.P. n. 25                    |  Loc. Pixina Manna Edificio 1
> 09010 Pula (Cagliari), Italy  |  09010 Pula (Cagliari), Italy
>
> Email: scheinin at crs4.it
>
> Phone: 070 9250 238  [+39 070 9250 238]
> Fax:   070 9250 216 or 220  [+39 070 9250 216 or +39 070 9250 220]
> Operator at reception: 070 9250 1  [+39 070 9250 1]
> Mobile phone: 347 7990472  [+39 347 7990472]
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>