[Beowulf] Heterogeneous, intermitent beowulf cluster administration
Gavin W. Burris
bug at wharton.upenn.edu
Thu Sep 26 06:25:42 PDT 2013
I'm a nay-sayer in this kind of scenario. I believe your staff time,
and the time of your lab users, is too valuable to spend on
dual-classing desktop lab machines.
If your lab is underutilized, I would spend staff time on figuring out
why and how to make the lab more effective as a destination for
prospective users. If you need more cluster compute time, I would
invest funds into additional compute nodes, not into micromanaging the
lab machines. Skilled sysadmin time is valuable.
Let's also consider the cost of electricity and cooling. I doubt that
the lab machines and climate control are the most efficient in terms of
full-throttle HPC/HTC computing. Electricity and cooling should be at
the top of your list for cost effective and green computing. I would
instead have the lab machines suspend/sleep until they require automated
patching or desktop login.
Also, let's consider the user experience. Cluster users will see jobs
killed and restarted; they will not be happy. Lab users will see slow
and/or hung machines; they will stop coming to the lab.
Don't get me wrong, this is an interesting project, but one riddled with
pitfalls. If the job is to support a computing lab, that should be goal
On Thu, Sep 26, 2013 at 10:00:28AM -0300, Ivan M wrote:
> Hi folks,
> I have access to a bunch (around 20) machines in our lab, each one with a
> particular configuration, usually some combination of Core i5/i7 and
> 4GB/8GB/16GB RAM (the "heterogeneous" part), connected by a 24 ports Cisco
> switch with reasonable backplane. They're end user machines, but with the
> current lab occupation only a fraction of them are used constantly, but
> which ones change every day. They are all running Debian stable. I got an
> idea: why not use the downtime to run some parallel simulations, instead of
> using the university cluster?
> They main problems now are:
> 1) System administration: for now I'm doing the clusterssh way to
> update/configure/install new software, but this can be very cumbersome, as
> one of the machines can be being used and so I can't change its
> configuration, so I have to keep track of which ones have changed. Maybe
> puppet can help here?
> 2) Managing resources: knowing which machine is up and available withou
> having to shout, and knowing the available configuration to allocate jobs
> that can fit in that particular machine, etc. There are extreme cases when
> the machine needs to be rebooted to run some Windows program.
> 3) Migrating jobs (the intermitent part): any machine can be requested by a
> user at any time, so if I have a parallel job running I would have to
> migrate the job to another machine, preferably without stopping the other
> jobs. We are running mostly ROMS over MPI and some in-house simulations
> that use a combination of OpenMP and MPI.
> Does anyone have any experience or pointers on how to address these issues?
> It seems a waste not to use those idle machines...
> Ivan Marinhttp://scholar.google.com.br/citations?user=faM0PCYAAAAJ
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Gavin W. Burris
Senior IT Project Leader
University of Pennsylvania
More information about the Beowulf