[Beowulf] quick and dirty method for starting job on another node?

[Greg M. Kurtzer]

On Tue, May 03, 2005 at 12:12:00PM -0400, Robert G. Brown wrote:
> You mean other than
> 
>   ssh -x targetnode /pathto/taskname args...
> 
> ???
> 
> (As in, doesn't your question contain its own answer?)
> 
> I enclose "taskmaster", a perl script I featured in one of my first CWM
> columns.  It has two nifty features that may be of use to you.  One is
> the "runtask" subroutine, which encapsulates the above for use in perl.
> 
> The other is the taskmaster main routine itself, which basically loops
> over hosts and spawns independent threads (perl now supports real
> threads) each of which contains an instance of runtask.
> 
> This isn't TOTALLY robust, but is pretty close and you could make it as
> robust as you like.
> 
> So one of these things ought to work for you -- ssh directly, runtask
> as a perl routine if the scripts your referring to happen to be in perl,
> a hack of taskmaster itself if you want the threads and greater
> robustness.  The threads are useful because of a change in ssh that
> makes it relatively difficult to disconnect an ssh session with a
> backgrounded task from its controlling tty to get back your shell.

Also don't forget about pdsh (http://www.llnl.gov/linux/pdsh/pdsh.html). The
2.x series handles either rsh/ssh at runtime, and is capable of specifying a
fanout of your choice. It also handles a pseudo interactive shell to multiple
nodes at the same time.

Pdsh is written in C so it can be used for minimal nodes, but note that its 
library requirements are non-minimal. I have not compiled this statically at
this time, because I don't run it on the nodes (just the master).

It also includes pdcp (parallel copy), which isn't near as fast as dolly but 
very convenient.

Hope that helps.
-- 
Greg Kurtzer
Berkeley Lab, Linux guy