[Beowulf] Announcing nettee 0.1.4
Greg M. Kurtzer
gmkurtzer at lbl.gov
Tue May 3 14:37:36 PDT 2005
Can you elaborate on the features that you have added to Dolly?
I worked on a project recently to handle system provisioning of +1000 cluster
nodes with 10GB file systems in under an hour. We used dolly in conjunction
with the Warewulf cluster manager utilizing 16-32 node segments for the dolly
ring and each ring having a "group lead"'s which pulls direct from the
Warewulf's VNFS (virtual node file system) daemon and then dolly's to the
other members of the group.
As this was just a proposal, we didn't have the resources to scale to 1000
nodes, but based on our tests it is theoretically very favorable that we can
do it in substantially under 1 hour. Dolly rocks (Felix++)! :)
On Tue, May 03, 2005 at 01:35:32PM -0700, David Mathog wrote:
> nettee is a network "tee" program derived from Felix Rauch Valenti's
> dolly program.
> nettee can create a network daisychain
> which passes a data stream along at full network bandwidth while
> (optionally) writing the data to each local node. If your disks
> are slow and the data larger than disk cache then it will only
> move data as fast as the disks can be written. Otherwise speeds
> up up to 11.3 Mb/sec were observed (100baseT). So nettee
> is useful for distributing large files or in conjunction with
> SystemImager to clone nodes.
> Additionally nettee can be configured to create a command chain
> that lets the same command run on every node in the chain
> with only a .01s delay from the time it is issued to the time
> it runs on all nodes (chain of length 20, 100baseT switched
> network.) The downside - stdout and stderr are usually lost
> or unusable.
> Example scripts showing a few uses for the program are
> provided in the distribution. Instructions for using nettee
> with SystemImager are found in the README.TXT file.
> Look here for more info:
> The program is still in alpha. It has been pretty reliable for me
> so far but your mileage may vary.
> And yes, I did look at netcat. In simple tests nettee moves
> large blocks of data about 25% faster than netcat does. I have
> not attempted to figure out why nettee is faster.
> David Mathog
> mathog at caltech.edu
> Manager, Sequence Analysis Facility, Biology Division, Caltech
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Berkeley Lab, Linux guy
More information about the Beowulf