[Beowulf] split traffic to two interfaces for two "subnets"

[Robert G. Brown]

On Wed, 10 May 2006, Yaroslav Halchenko wrote:

> Hi All Beowulfers,
>
> Please kick me if this is a basic networking question and I must have
> known it from my kinder-ages
>
> I got a new file-server (SATA on top of Areca card, 6TB, < 8,000$),
> which seems to perform quite well under some preliminary tests and it is
> capable to saturate the network very easily.
>
> I have 25 nodes all on 10.0.0.0/24 network, new file-server (lets call
> it raider) has 2 gigabit ports. I would like to spread the I/O load so
> that
>
> 10.0.0.16/28 goes through eth1
> the rest 10.0.0.0/24 can go through eth0 (as before)
>
> It seems to be quite easy: just set eth1 explicitly with /24 network
>
> Now on 10.0.0.16/28 I need to nfs mount from the ip associated with eth1
> (named raider1) and that seems it.
>
> Did I miss something? may be raider would get confused since there would
> be overlapping networks on different interfaces and 10.0.0.31 will now
> be an broadcast (I don't utilize that IP anyways for now but might in
> the future)

Since you have an "infinite" IP space it seems like there are several
ways to skin this cat.  For example you might use 10.0.0.x for 13 nodes
+ eth0 and 10.0.1.x for 12 nodes + eth1 and route accordingly.  It's the
routing (mostly from the server to the nodes) that is the tricky part,
not the two interfaces per se.  This might make it a bit easier to
segregate the broadcasts.

I think this is pretty much what you are trying now except that instead
of screwing around with binary masks and broadcasts at the bit level,
where it is pretty easy to make a mistake, you can do it at the byte
level.  You basically can't run out of addresses no matter what you do,
so why be conservative with them.

> Or may be there is a more elegant solution? (virtual server...)

If you have a smart enough switch maybe.  Or if you have e.g. myrinet
for IPCs and are using the ethernet only for shell and nfs type services
you can just split the two interfaces (and their nodes) and put them on
two different switches as well as use two class C subnets.  That will
REALLY split the traffic -- broadcasts and so on will be appropriately
segregated by the switch boundaries.  This is "bad" as far as node
10.0.0.4 talking to node 10.0.1.5 (on two different switches) -- they
would have to talk through e.g. the server configured to route/forward
the packets.  But if you never need for a node on one switch to talk
efficiently to a node on the other switch via TCP/IP, why not?

   rgb

>
> Thank you in advance
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu