[Beowulf] clustering using xen virtualized machines

[Jonathan Aquilina]

thanks for all yoru responses. i admit i dont have the money at the moment
or a job to get my hands dirty with hpc. im planning in the future to setup
a rendering cluster. i appreciate all the feed back here.

im just wondering now would for instance a head node be of any use running
virtualized guest os's or does the head node need to not share the hardware
with other os's

On Wed, Jan 27, 2010 at 4:18 PM, Gavin Burris <bug at sas.upenn.edu> wrote:

> The cost for virtualization is in buying really big hardware, oodles of
> memory and many many cores, that are capable of running multiple VMs,
> and having that hardware configured for redundancy, high availability
> and failover.
>
> With an HPC cluster, you are typically buying hardware that is as
> stripped down and cheap as you can get it.  You focus your HPC budget on
> the sweet-spot processor, the amount of memory, maybe GPUs, maybe
> interconnect, so you can deploy as many compute server nodes as you can
> afford.
>
> I don't buy the argument that the winning case is packaging up a VM with
> all your software.  If you really are unable to build the required
> software stack for a given cluster and its OS, I think using something
> like xCAT to provision stateless compute servers per job is a better
> option than virtualization.
>
> And if you are packaging VMs to blast out to the cloud, I think you will
> be paying through the nose.  This is not a viable option unless there is
> a major pricing shift.
>
> Cheers.
>
>
> On 01/27/2010 07:08 AM, Jonathan Aquilina wrote:
> > gavin you mentioned costs, those are only incurred with xen if you need
> > the extra features such as server migration and other features. also if
> > you dont need those extra features couldnt you just live with the free
> > version of xen.
> >
> > On Wed, Jan 27, 2010 at 11:42 AM, Geoff Galitz <geoff at galitz.org
> > <mailto:geoff at galitz.org>> wrote:
> >
> >
> >
> >     I've had the good fortune to be in the HPC and also HA business for
> >     a few
> >     years (10 years for HPC but only about 4 for HA).  Given the current
> >     approach for virtualization I don't see that Xen or other
> virtualization
> >     technologies are good for HPC environments if the performance is a
> >     paramount
> >     concern.
> >
> >     Virtualization in an HPC/HA world is mostly beneficial for
> >     portability and
> >     fail-over.  But the added layer for a hypervisor will be significant
> >     if your
> >     jobs run for an extended period of time.  I've seen jobs that run for
> >     months...  a 7% performance penalty (fairly typical in my
> >     experience) over
> >     the course of a month is significant.
> >
> >
> >
> >     ---------------------------------
> >     Geoff Galitz
> >     Blankenheim NRW, Germany
> >     http://www.galitz.org/
> >     http://german-way.com/blog/
> >
> >
> >
> >     _______________________________________________
> >     Beowulf mailing list, Beowulf at beowulf.org
> >     <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
> >     To change your subscription (digest mode or unsubscribe) visit
> >     http://www.beowulf.org/mailman/listinfo/beowulf
> >
> >
> >
> >
> > --
> > Jonathan Aquilina
> >
> >
> >
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
Jonathan Aquilina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20100127/b41d292b/attachment.html>