[Beowulf] using two separate networks for different data streams

[Dan Stromberg]

On Fri, 2006-01-27 at 19:57 +0100, Daniel Pfenniger wrote:
> 
> Ricardo Reis wrote:
> > 
> > First, Hi all and thanks for your answers. Were truly useful. Which
> > brings me to...
> > 
> > On Fri, 27 Jan 2006, Mark Hahn wrote:
> > 
> >> I wonder whether anyone has critically evaluated whether this is
> >> important.
> >> cluster people I talk to like to say fuzzy things like "separate networks
> >> make the cluster breathe better".
> >>
> >> as much as I admire car analogies, I observe that when apps are doing IO,
> >> they tend not to be doing MPI.  if your workload is like that, bonding
> >> rather than partitioning would actually improve performance.  I wonder
> >> whether the partitioning approach might actual reflect other constraints,
> >> such as using half-duplex hubs, or low-bisection networks.
> 
> The network for MPI should in many cases have low latency, so is expensive
> (Myrinet, InfiniBand, etc.) in regards of Ethernet.  The I/O, NFS and
> system network does not need low latency, and so for bargain cost can be
> added, with the additional ground that it provides a control network to
> tweak the nodes remotely when the expensive low latency network is down.

That leads to a question for the compute cluster we're currently
planning to buy here at UCI:

Is there a way of characterizing in what proportion a given application
relies on OpenMP, and how much the application depends on MPI (and hence
MPI network latency) - other than speaking with application developers
to get their intuitive feel, that is?  :)

We're looking to buy a Gigabit Ethernet network for the MPI on this, but
if that's obscenely high latency, and the primary application the
cluster's being purchased for is heavily dependent on MPI, then we might
want to be ignoring the GigE and going for something else.

Any thoughts?

Thanks!