[Beowulf] quick and dirty method for starting job on another node?
Greg M. Kurtzer
gmkurtzer at lbl.gov
Tue May 3 13:44:53 PDT 2005
On Tue, May 03, 2005 at 12:12:00PM -0400, Robert G. Brown wrote:
> You mean other than
> ssh -x targetnode /pathto/taskname args...
> (As in, doesn't your question contain its own answer?)
> I enclose "taskmaster", a perl script I featured in one of my first CWM
> columns. It has two nifty features that may be of use to you. One is
> the "runtask" subroutine, which encapsulates the above for use in perl.
> The other is the taskmaster main routine itself, which basically loops
> over hosts and spawns independent threads (perl now supports real
> threads) each of which contains an instance of runtask.
> This isn't TOTALLY robust, but is pretty close and you could make it as
> robust as you like.
> So one of these things ought to work for you -- ssh directly, runtask
> as a perl routine if the scripts your referring to happen to be in perl,
> a hack of taskmaster itself if you want the threads and greater
> robustness. The threads are useful because of a change in ssh that
> makes it relatively difficult to disconnect an ssh session with a
> backgrounded task from its controlling tty to get back your shell.
Also don't forget about pdsh (http://www.llnl.gov/linux/pdsh/pdsh.html). The
2.x series handles either rsh/ssh at runtime, and is capable of specifying a
fanout of your choice. It also handles a pseudo interactive shell to multiple
nodes at the same time.
Pdsh is written in C so it can be used for minimal nodes, but note that its
library requirements are non-minimal. I have not compiled this statically at
this time, because I don't run it on the nodes (just the master).
It also includes pdcp (parallel copy), which isn't near as fast as dolly but
Hope that helps.
Berkeley Lab, Linux guy
More information about the Beowulf