[Beowulf] One network, or two?

[Joshua Baker-LePain]

On Tue, 23 Sep 2008 at 9:16pm, Alan Ward wrote

> I have been reading the ongoing discussion on network usage with some 
> interest, mainly because in all (admittedly very small, 4 to 8 node) 
> clusters we have set up so far, we have always gone with doubling the 
> network. Nowadays we mostly run a 100 MBit/s "el cheapo" FastEthernet 
> for control, NFS and monitoring, while the faster Gigabit is exclusively 
> for MPI. Applications are CFD, with various levels of granularity.

In theory, it's a great idea.  In practice, wiring up an entire 2nd 
network gets to be a major PITA (as well as being uncheap) as your cluster 
size increases.  I've got ~350 nodes here, and a 2nd set of cabling is 
really, really low on the list of projects I'd like to tackle.

-- 
Joshua Baker-LePain
QB3 Shared Cluster Sysadmin
UCSF