[Beowulf] High Performance SSH/SCP

[Robert G. Brown]

On Fri, 15 Feb 2008, Prentice Bisbal wrote:

> 4 years ago, I set up kerberos for the very first time, without any
> prior experience. I read through the relevant chapters of the O' Reilly
> Kerberos book and had it up and running in only a couple of days. Most
> of that time was spent reading. I disagree with RGB's equation
>
> rsh + anything = (difficulty)^2
>
> I found that once kerberos is set up, using kerberized rsh is
> essentially invisible, therefore
>
> rsh + kerberos = difficulty
>
> which is a first-order relationship.
>
> If anything, setting up rsh is the most difficult one. Why? Since rsh is
> so insecure, the distro producers/vendors have created many hurdle you
> must hop to get it working (correct file and owner permissions, etc.)
> Ironic.

My equation was empirical, and it isn't really squared, and the
experience was with real newbies.  You have to understand, a lot of
people who get into cluster computing really have NO zero nada Unix
experience.  Some of them ask me how to set up a Windows cluster
initially, until I point out that I don't do Windows (and here's why).
I summarized that a bit in the last post, though, and won't
recapitulate.

Also, things just plain don't always work.  Not even for me on my own
systems, and I'm a 22 year Unix sysadmin (just kidding, last post, about
the amateur bit:-).  Just as a currently relevant example:

(Following yum install \*pvm\*, Fedora 8, same laptop on which pvm
WORKED when I was asking about it last week.)

...
Total download size: 2.6 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): pvm-3.4.5-7.fc6.1. 100% |=========================| 2.1 MB 00:02 
(2/2): pvm-gui-3.4.5-7.fc 100% |=========================| 549 kB 00:00 
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
   Installing: pvm ######################### [1/2]
   Installing: pvm-gui ######################### [2/2]

Installed: pvm-gui.i386 0:3.4.5-7.fc6.1
Dependency Installed: pvm.i386 0:3.4.5-7.fc6.1
Complete!
rgb at cain|B:1031#z

[3]+  Stopped                 su
rgb at cain|B:1038>xpvm
libpvm [pid14741] /tmp/pvmd.1337: No such file or directory
libpvm [pid14741]: Can't Start PVM: Can't start pvmd

OK, this is trying to start up xpvm (or pvm) on a brand, shiny,
so-new-it-hurts fresh yum install on Fedora 8 on a system where it
worked last week. It quit.  I haven't GOTTEN to where rsh or ssh matters
(and besides, I have had a working PVM environment for about fifteen
years now).  So what's wrong?

If this isn't bad enough, I installed PVM two days ago on nine F8 boxes
and IT worked perfectly, but xpvm failed (and I discovered that it
wouldn't work on my laptop where it was just working).  Why?  It will no
doubt be a long and painful process to figure it out.  To first order,
the PVM RPM installs the binaries in bin/LINUX under intel, but the
/usr/bin/xpvm script is forming LINUX$(ARCH) and it should be LINUXI386
-- this is an actual PVM bug -- but there is still more wrong.

Now imagine me helping a newbie get started with PVM.  I've used it for
years, works perfectly.  They tell me that they run pvm and get an error
message like this one.  EVEN if I've seen this before and figured it out
once, I've long since forgotten.  It has re-broken.  Or it's new.  Is it
PVM?  Is it paths?  Is it rsh or ssh?  Bug in the distro RPM?

When things work like they're supposed to, it is as you say extremely
simple and the online guides work fine.  When it doesn't, you have to
debug EVERY ERROR PATHWAY, and adding layers adds infinite pain.
Infinite pain "squared", if you're trying to do so via email with
somebody on the other end of the line that is using an xterm for the
first time and who might be working as root for all that you know.

So sure, as I said one can certainly do what you describe if you know
what you're doing, are self-starting, a manual reader, and a problem
solver.  YMMV painwise, even if you're an expert, depending on whether
things WORK they way they are supposed to and how hard it is to discover
where your setup deviates from the assumptions in the documentation.
But if you fail on any of these, the pain index rapidly escalates to
"quit and write novels instead", or "get new job as prison guard" or
even "perform self-appendectomy with a rusty spoon".  (You laugh, but
I'd bet even money that a show of hands among the real old hand cluster
people on list would turn up a dozen or more with some nasty scars on
their lower abdomen;-)

    rgb

>
> - --
> Prentice Bisbal
> Linux Software Support Specialist/System Administrator
> School of Natural Sciences
> Institute for Advanced Study
> Princeton, NJ
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFHtgoh2n4m8G8ypgARApKbAKDTM4dDBTbEsPGZ96ATimhh93akEACgqhfW
> XKuset0dI1xR9rAq3OY38fM=
> =KQGF
> -----END PGP SIGNATURE-----
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977