network profiling

[Robert G. Brown]

On Thu, 8 Feb 2001, J.   RAHEB wrote:

> Hello everyone,
>
> I was wondering if anyone knew of a program that can profile your BEOWULF
> network usage while applications are running.  There might actually be one
> included in RedHat, but I'm not sure.  If anyone knows of one I would
> appreciate the info.

To a certain extent, network usage is provided in real time in
/proc/net/dev.  For example:

rgb at ganesh|T:486>cat /proc/net/dev
Inter-| Receive | Transmit
 face |bytes packets errs drop fifo frame compressed multicast|bytes
packets errs drop fifo colls carrier compressed
    lo:121838079 265657 0 0 0 0 0 0 121838079 265657 0 0 0 0 0 0
  eth0:1714321879 39538698 0 0 11 0 0 0 1152059775 29745921 0 0 0 0 1 0

Note that this counts both bytes and packets both sent and received, as
well as errors.  Those numbers can easily be parsed with e.g. a perl
fragment or a c fragment.  The perl I leave as an exercise; the c code
needed to parse it (by interface) is in the procstatd package available
on www.phy.duke.edu/brahma.  Indeed, procstatd would let you just read
off the packet traffic in real time, if you just want to see/sample the
load at certain points in a run and not record it over some hours.

So one way to do it would be to write a simple loop perl script that
parses out bytes sent and bytes received on eth0 at some time, creates a
delta from the previous time, sleeps a time, and does it again and
again.  The output delta is written to a file in LOCAL disk /tmp if
possible, or written to a screen, or whatever you like.  Be aware that
if the program itself utilizes the network (to write its results to an
NFS file system, or to send them back to an xterm on a remote host) then
there will be a certain error associated witht he measurement process
itself, and take care to keep the error small compared to the
application-based traffic.

If all you care about are aggregates, you could have the same perl
script start and stop the counter (and create the delta) and invoke the
program itself.

Anyway, you get the idea.  It really is pretty simple to do if you know
perl.  If you don't, it is an amusing enough project I might do it for
you (or have a student do it for you).

   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu