The FNN (Flat Neighborhood Network) paradox

[Robert G. Brown]

On Wed, 21 Feb 2001, Jim Lux wrote:

> <nitpick>
> A tetrahedron isn't a hypercube, is it?  A 4 node hypercube would have 2
...
> </nitpick>
<smile>Got me...</smile>

> >A five port switch costs perhaps $70 (or less if you shop hard) -- eight
> >port switches are as little as $80.  A switched port costs LESS than a
> >NIC these days.  Admittedly these switches are likely to be
> >store-and-forward with mediocre latency, but even better switches aren't
> >that expensive anymore.  Add in only FOUR NICs and cables @$25 each, and
> >you can get effortless connections for only $170 and have an extra port
> >to connect up a head node or to another switch.
>
> What are cheap/inexpensive switches that don't use store and forward?

I meant cheap switches that still use store and forward but have
somewhat better latency than the really cheap ones.  Poorly worded,
sorry.  All I meant is that I have no idea what the latency is going to
be on an $80 switch, but I'll bet it isn't particularly good (it isn't
on my at-the-time-cheapest-switch).  It's probably worth at least trying
to look at the latency numbers on switches with comparable numbers of
ports but wildly different prices and seeing if the more expensive
switches deliver better latencies, if latency matters in your
application.

> This discussion brings up an interesting question:
>
>
> Say you had 2 8 port switches and wanted to interconnect, say, 8<=N<=4
> processors with a pair of NICs in each.  What is the optimum arrangement for
> channel bonding?  All processors send one NIC to one switch and the other to
> the other?, or some sort of hybrid where you send both NICs to the same
> switch. (obviously, one can scale this problem up... in today's market, the
> correct answer would be to just buy a 16 port switch).  How well does

I think that this is really the right answer though.  Just buy a bigger
switch until you cannot afford anything any larger, and long before that
time you'll likely have moved over into the regime of either GB ethernet
or Myrinet or out in the esoteric region where the answer requires
serious math to work out and is likely different for different kinds of
network access.  I'd guess the >>optimum<< answer would be different for
a parallel process with short range (one message to "nearest neighbors"
with some topology), long range but symmetric (one message to N hosts)
and long range buy asymmetric (N different messages to N hosts, and that
the optimum answer starts to become very expensive regardless, so for
big designs you really have to just work it out.

   rgb

> channel bonding deal with the potentially different latency through
> different paths?  Does the various protocols deal well with messages
> arriving out of sequence? Obviously, some layer of the protocol could be
> responsible for buffering and rearranging, but that might not be the best
> approach for some applications.
>
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>
   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu