software running on the nodes?

[Robert G. Brown]

On Tue, 14 Jan 2003, Mike Eggleston wrote:

> Has this been asked for? Can we have a quick, informal poll
> on what os/distribution/rpms/etc is running on people's nodes?

RH 7.3, custom kickstart installation.  Several hundred nodes (and lan
workstations) in our department, but there are other departments who may
or may not be using the same Dulug Beowulf group below as there are lots
of clusters on campus now.  Mostly dual nodes, both PIII and Athlon.

See below (with encrypted root pw edited out:-) our "beowulf" ks file,
and below it the relevant additions to the standard comps file to define
the dulug beowulf group.  Pretty straightforward -- sundry math/stats
including blas/lapack, lam, pvm, gsl, xmlsysd/wulfstat to facilitate
node monitoring, and a workstation-like install elsewhere.  We (re)wrap
everything local including proprietary site-license software in rpm form
and restrict access accordingly.  Perhaps this is a bit fatter than
strictly necessary (installing X just so one can run x-tools on a
node:-) but disk is "infinite" these days, our department network is
basically flat wrt the cluster, and so who cares?

We have a second image that doesn't create a rest-of-the-disk xtmp
directory for use on small disk systems.  Note that 3 GB is plenty for
our OS image at least so far, and having a rest-of-disk, non-backed-up
xtmp on each node provides many GB of scratch space for users without
needing to know much about the node's specific disk other than it is
bigger than 5 or 6 GB.

There is also sundry magic worked in the post -- security, access, grub,
and more.

Note that a lot of this is due to Seth Vidal, our local linux genius who
has set up and runs the dulug server(s).  Anybody on campus (with a
duke.edu address:-) can install linux consistently in a manner of
minutes, automagically updated to the latest updates nightly via yum.
This supports undergrads and grads, individual faculty at home (with
duke.edu or proxy/tunnel) or in departments, department admins for their
department lans, cluster owners and their clusters.  Anybody on campus
can install using the dulug beowulf group (and their own %post and
additions/subtractions).  Any admin who learns to use kickstart, grub,
dhcpd and pxe can install or reinstall any cluster, cluster node, PC or
workstation in a matter of minutes per node, where most of that time is
spent waiting, not working.  

Just one person (Seth) does ALL the toplevel grooming of the actual
distributions and support of the requisite tools.

This scales pretty close to the theoretical maximum as a consequence,
especially given that Seth's primary job ISN'T maintaining this, but
running the physics department LAN.  One fraction of an FTE (admittedly,
a cosmically >>talented<< FTE:-) can and does maintain the entire tree
structure that supports an entire University's use of linux -- thousands
of systems -- including the completely automated distribution of e.g.
security or functional updates as they are released.  Seth actually
provides a lot of detailed support for individual users and other linux
admins via the dulug mailing list, as well, although there he has at
least some help from the rest of the expert user base.  All this and a
real job too, gee!

I'd like to see a single MCSE attempt this.  Ever.  Makes me want to
laugh hysterically just thinking about it -- tracking the licenses alone
for 1000+ systems would eat an FTE, and then, one has to pay for all
those licenses.  With Win(X < 2000) I think it would be openly
impossible.  With Win2000, there are at last some tools in MS for remote
installs, but I suspect that server scaling and rising costs would
utterly destroy you long before you could install a single 100+ node
cluster, let alone a dozen LANS from a single common webserver.

   rgb

Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu

%< Snip snip snip =====================================================
install
text
lang en_US
langsupport en_US 
keyboard us
mouse none
network --bootproto dhcp
rootpw --iscrypted dontyouwishdontyouwithdontyouwish
url --url http://install.dulug.duke.edu/pub/linux/dulug-7.3/i386
timezone US/Eastern
skipx
auth --useshadow --enablemd5
bootloader --location=mbr
zerombr yes
clearpart --all
firewall --disabled
part / --size 3000 --fstype ext3
part swap --recommended
part /disk1 --size 2000 --fstype ext3
part /xtmp --size 1000 --grow --fstype ext3
reboot

%packages
@ Dulug Beowulf
amanda
amanda-client
aspell
htmlview
ImageMagick
joe
jove
mgetty
openssh-server
postfix
pspell
pump
rdate-cron
symlinks
vim-enhanced
wget
xinetd
-cups-drivers-hpijs
-cups-drivers-pnm2ppa
-cups-drivers
-dateconfig
-dhcpcd
-ghostscript
-ghostscript-fonts
-gv
-hpijs
-logwatch
-ntp
-Omni
-Omni-foomatic
-printconf
-printconf-gui
-rhn_register
-rhn_register-gnome
-sendmail
-sendmail-cf
-sudo
-up2date
-up2date-gnome

%post
cd /tmp
/usr/bin/wget -q -nd -nH http://install.phy.duke.edu/rh-7.3/post/beowulf.sh
chmod +x /tmp/beowulf.sh
/tmp/beowulf.sh 2>> /tmp/post-errors.log
rm -f /tmp/beowulf.sh



%< Snip ditto ------
0 Dulug Stat/Math {
 ? X Window System {
     octave
     gnuplot
     root   
     root-examples
     root-documentation
     postgresql-libs   
     R-base
     R-recommended
 }
 gsl
 gsl-devel
 hpc
 blas
 blas-man
 lapack  
 lapack-man
}

0 Dulug Beowulf Misc {
  xmlsysd
  wulfstat
  ntp
  pvm
  pvm-gui
  lam
}

0 --hide Dulug Beowulf {
 @ X Window System
 @ Network Support
 @ Utilities
 @ Emacs
 @ Software Development
 @ Dulug Stat/Math
 @ Dulug Beowulf Misc
}