Beowulf and variable cpus

[Drake Diedrich]

On Thu, Sep 21, 2000 at 12:49:46PM -0400, Troy Baer wrote:
> On Thu, 21 Sep 2000 p.grimshaw at virgin.net wrote:
> > Hi, I am new to Beowulf and have some questions,
> > 
> > 1. Does anyone know if I am able to run a beowulf cluster with
> > different types of clients, i.e I have a load of pentium 100s
> > and some p2 500s which I would like to use together. Is this
> > possible?
> 
> Sure, it's definitely possible.  You may run into load balancing problems
> if you have a parallel programs running on both fast and slow CPUs.  If
> you use a batch system like PBS, you can create attributes for individual
> nodes and use those to ask a homogeneous set of processors (or
> heterogeneous, for that matter):
> 
> # ask PBS for 10 Pentium 100MHz nodes
> #PBS -l nodes=10:p_100
> 
> # ask PBS for 6 Pentium II 500 MHz nodes
> #PBS -l nodes=6:p2_500
> 
> # ask PBS for 10 Pentium 100s *and* 6 P2 500s
> #PBS -l nodes=10:p_100+6:p2_500
> 
> I'd assume/hope you can do similar things in other resource managers and
> batch systems, like LSF and DQS (although I've never tried either).
> 

   It's very similar under DQS (also aims for Posix batch system compliance).
#? is the in-job DQS flag delimiter.  -l to specify resources.  Other than
qty most resources are sysadmin-defined in queue complexes (qconf -ac
complexname) and then associated with real queues (qconf -cq/-mq).
  Some examples in the DQS qsub manpage:

# I want 6 or more machines of group ibm
# with memory greater than 128M, that has pvm
#$ -l qty.gt.6,mem.gt.128,group.eq.ibm,pvm

# If I can get it, I would also like 1 machine
# with memory greater than 256M of group dec
#$ -soft -l qty.eq.1,mem.gt.256,group.eq.dec

   Resource flags may use either Fortran or C-like operators for comparison.

   Our group has a small cluster with 4 completely distinct architectures, 3
different operating systems, and differing memory and processor
counts/speeds on nearly every node.  Major application software (licensed or
otherwise available on only some machines) are listed in the queue complexes
allowing users to specify required programs as just another resource.  Most
jobs here are serial or embarrasingly parallel, so the inconvenience of load
balancing heterogenous sets of machines isn't an issue.  Homogenous subsets
could be selected using resources limits if necessary.
   The nodes assigned to a job are listed in a file pointed to by the
HOSTS_FILE environment variable.  A PVM virtual machine may also be set up
for the job by DQS if an appropriate flag is set (-par PVM), though there
are some difficulties if interactive jobs or multiple jobs for the same user
on the same nodes are also setting up PVMs.

-Drake