[Beowulf] PVM on wireless...

[Robert G. Brown]

On Thu, 7 Feb 2008, kohlja at ornl.gov wrote:

>  > It would really, really help if man pvm (or man pvmd or man pvm_intro)
>  > documented a suitable firewall setting that will let PVM function
>  > without just turning off the firewall altogether.  There is no pvm setup
>  > in /etc/services, for example, no pvm checkbox in the panels managed by
>  > system-config-firewall in the latest Fedoras, no suggestion as to what
>  > what protected port(s) or ranges one has to enable explicitly.  In fact
>  > for once even google is failing me -- I'm not finding a lot of
>  > documentation or remarks by ANYONE on what ports pvm needs open (besides
>  > ssh, which obviously is open and works).  Usually as long as the
>  > spawning of a network application itself works using an enabled
>  > protected port (in this case, I would have expected ssh), the secondary
>  > ports opened in unprotected space just work.  Am I wrong in this?  Do I
>  > need to explicitly open more ports somewhere?
>
> Ah Yes.  O.K., so I wish it was that simple, but alas PVM can use as
> many ports as you have machines in your cluster, or could use just 1.  :-}
>
> Normally, the master pvmd creates/accepts connections over a small
> set of ports, possibly 1, but if PvmRouteDirect is enabled in a PVM
> application, then a myriad of direct-connection socket links are
> created, to link whichever machines the local PVM application tasks
> communicate with, on a demand-driven basis...
>
> So it's not generally possible to specify an explicit "range" of ports.
> However, it _is_ possible to set the "starting" port for this collection,
> using the aforementioned "$PVMNETSOCKPORT" environment variable.

OK, I'm giving this a try.  Although I'd have to ask why pvmd doesn't do
the fork thing and clone a single open port on which it listens into a
dynamically allocated port that inherits from the open one.  In
principle one only needs a single port to be open to connect to pretty
much any network based application, or so I had thought.  At least, I do
that in xmlsysd and never have to punch more than one porthole through a
firewall.

Hmmm, it's working sort of -- looks like I need to open UPD ports,
right, not TCP?  Having trouble on one host where I've punched the hole
but didn't >>locally<< set PVMNETSOCKPORT to match, so I'm trying again
with the local environment variable set.

Yup, that works.

So I'm guessing that pvmd reads it as it starts up wherever.  Why does
it need to do this on a client?  Can't the port(s) be passed from the
master when it starts up pvmd?

> This sets the first port that PVM will try to use, and all subsequent
> ports will usually be consecutive positive increments of that starting
> port (i.e. PVMNETSOCKPORT++... :-).
>
> So in most cases, you could probably plan on opening up a 100 or 1000
> ports _somewhere_ in your firewall, depending on your needs, and then
> just tell PVM where to start, using $PVMNETSOCKPORT...
>
> I've always considered this solution a bit of a kludge, which is why
> it doesn't show up in the man pages, but if it works well enough to
> save people lots of hassle, then I can add some commentary on it...?

Kludge or not, how can you have an environment variable in an
application and not provide knowledge of it or instructions on its use
in the man page?  Something like:

  PVM requires open ports on target hosts to function.  Many hosts are
  installed with strong firewall rules by default.  If you install pvm on
  a slave and pvm appears to hang when you attempt to add it, eventually
  timing out without success, consider adding the following to your local
  personal or system environment (in, for example, ~/.bash_profile on all
  hosts):

    PVMNETSOCKPORT=10000
    export PVMNETSOCKPORT

  Then configure your firewall(s) to open a range of udp ports starting
  at this value, such as 10000-11024 (which need be any larger than the
  largest number of machines you expect to have in your virtual machine).

However a better solution still is to have the daemon fork on a single
"permanent" port address > 1024, e.g. 10000, and get a negotiated
connection in the upper (non-protected) port space that way.

> It may depend on the firewall settings, but a nice "Connection
> Refused" would usually go a long way toward diagnosing things,
> whereas the more secure firewall alternative of simply
> "no response" would only result in a "timed out" PVM message...
>
> I'm open to suggestions on ways to identify or diagnose the problem...!

As I said, document EVERYTHING in the man page(s).  It is what it is for.
Lots of users do, in fact, RTFM but get frustrated and give up when they
try something and it just doesn't work and they can't see why.

On the same line, a perennial problem with PVM is getting it to work
with rsh and ssh.  In fact, half the problems I help people with who
randomly write me is getting it to work with one or the other.  The
internal diagnostics are certainly very helpful, at this point, but it
would also be worth adding a new man page like pvm_rsh that does nothing
but walk users through the ritual of setting PVM_RSH and establishing
appropriate e.g. ssh keys.

Just a thought or two.

    rgb

>
> Thanks Much for your interest and feedback!
>
> All the Best,
>
> 	Jeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeem ;)
>
>  > I actually help a lot of people get started with PVM (they write me
>  > offline because I have a template PVM tarball up on my personal website)
>  > and the more I know, the better I can help them...;-)
>
>  >    rgb
>
>  > --
>  > Robert G. Brown                            Phone(cell): 1-919-280-8443
>  > Duke University Physics Dept, Box 90305
>  > Durham, N.C. 27708-0305
>  > Web: http://www.phy.duke.edu/~rgb
>  > Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
>  > Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977
>
> (:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:(:
>
>   James Arthur "Jeeembo" Kohl, Ph.D.     "Da Blooos Brathas?!  They
>   Oak Ridge National Laboratory              still owe you money, Fool!"
>   kohlja at ornl.gov
>   http://www.csm.ornl.gov/~kohl/          Long Live Curtis Blues!!!
>
> :):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):)
>

-- 
Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977