[Beowulf] 1 multicore machine cluster

[Gus Correa]

Glen Beane wrote:
> 
> 
> 
> On 4/21/09 3:46 PM, "Jonathan Aquilina" <eagles051387 at gmail.com> wrote:
> 
>     is it possible to have a single multicored machine as a cluster?
> 
> 
> 
> That wouldn’t exactly be a cluster, would it?  But you can certainly run 
> a lot of the software associated with Beowulf clusters: a batch system 
> (TORQUE, SGE, etc), MPI, ... so in practice you can use your 8 core 
> workstation just like you would a cluster.
> 
> 
> -- 
> Glen L. Beane
> Software Engineer
> The Jackson Laboratory
> Phone (207) 288-6153
> 
> 

Hi Jonathan, Glen, list

Along the lines that Glen pointed out,
I setup a dual-socket dual-core workstation here with OpenMPI
and MPICH2, plus Torque, to run some of our atmosphere modeling code
in batch mode.
It is not really a cluster, but a workstation
with some software characteristic of a cluster.

We tend to have long series of atmospheric model runs, where
each one-year simulation restarts from the previous state where
the last run stopped.
Each run can take, say, half a day to complete,
and the whole series may take a week to a month to finish.
Queuing the jobs up on Torque/PBS,
and forgetting about them until the whole series is done is
very convenient.

This setup works fine as long as the workstation is relatively idle.
However, if/when the owner decides to run heavy data analysis
Matlab scripts interactively while the MPI jobs are running,
then we get to memory contention, swapping, and all those bad things
that kill performance and may even break MPI jobs.
This "time shared" interactive activity, that is typically absent in 
cluster nodes, is germane to workstations.

Fortunately, I could convince the workstation owner (who also wants
the output of the atmospheric model runs) to do heavy interactive
use only when there aren't jobs on the Torque queue.
Or to suspend the job queue, wait for running jobs to complete,
work interactively, then restart the queue.

You can think of other heavy interactive use (e.g. streaming video)
that can produce the same negative impact on MPI jobs,
and you may need to adopt a similar
policy to avoid conflict between interactive and batch use
in your workstation, if you set it up "as a cluster".

My two cents.

Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------