[Beowulf]Infrastruture planning for small HPC 40/100 gigabyet eyhernet or Infiniband?

[Jim Lux]

Quoting "Robert G. Brown" <rgb at phy.duke.edu>, on Tue 29 Jul 2008  
04:46:30 AM PDT:

> On Mon, 28 Jul 2008, Gerry Creager wrote:
>
>> Back when the Earth was young, and the crust was still cooling, we   
>> ran serial connections between computers, over long distances and   
>> sometimes between power distributions.  It wasn't uncommon to see   
>> ground loops lead to arcing. I don't see it as much now because I'm  
>>  a little more careful about my grounds, and I bridge such problems  
>>  with glass rather than copper.
>>
>> The potential is still very real.
>
> The potential is very real, and even if the wires at both ends are
> "supposed" to not be touching anything even as "neutral" as the case
> ground, given the number of machines with network interfaces made by
> small shops in taiwan or the phillipines out of a stock chip but with
> their own local design team, who can doubt that there are ones where
> they do?  Ground loops are generally murphy's law objects, and since
> they CAN happen, sooner or later they will.


Actually, if they're following the RS-232 spec, it WILL occur, because  
that spec requires, for instance, that pin 1 of the DB25 connector be  
connected to the chassis.  If that chassis is connected to electrical  
safety ground (green wire/third prong) (as it should be), and the two  
ends are in different buildings, fed by different systems, the  
liklihood of significant potential difference is pretty high.   
(differentiate between pin 1, chassis ground, and pin 7, signal ground  
).  It's also why the RS232 spec calls for +/- 3V as the "deadband" in  
between the two levels.


But, RS232 was never intended for distances over, say, 10 meters.  
That's what short haul modems were all about... (basically line  
drivers/receivers with galvanic isolation)

Just this sort of thing is why the very first Ethernet (before it was  
even called that) called for galvanic isolation in the AUI.

This is also why there are grounding rules in the National Electrical  
Code, especially dealing with connections between buildings/structures.