[Beowulf] clustering using xen virtualized machines

[Tim Cutts]

On 28 Jan 2010, at 3:10 pm, Mark Hahn wrote:

>> I don't buy the argument that the winning case is packaging up a VM  
>> with
>> all your software.  If you really are unable to build the required
>> software stack for a given cluster and its OS, I think using  
>> something
>
> you're right, but only for narrow-function clusters.  suppose you  
> have a cluster used by 2k users across a handful of different  
> universities
> and 100 departments.  and have, let's say, 2 staff.  it's conceivable
> that using VMs would permit a higher level of service by putting  
> more configuration flexibility into the hands of the users.  yes,  
> most would
> use a standard image (which might be the bare-metal one, actually),
> but making it easier to accommodate variance is valuable.
>
> it even offers the ability to shift the model - instead of actually  
> booting VMs on nodes for a job, how about just resurrecting a number
> of VM instances (freeze-dried in already-booted state)?  that makes  
> the setup latency potentially much lower.  (pages from a VM image can
> be fetched lazily afaik, and presumably also COW.)

COW is certainly how some of the virtual desktop solutions work;  
desktop machines are 90% identical in most organisations, so it makes  
sense to use COW when firing up a new one.  So the technology is  
definitely around.

> for the few HPC-oriented performance studies of VMs I've seen,
> the only slowdowns were for OS activity (IO, page allocation, etc).
> an ideally-behaved HPC app minimizes those already, so...

We've certainly seen some interesting behaviour as far as the network  
is concerned.  We tried creating a VM with a Lustre client in it, and  
have not had much success with that.  There's more variability in  
network latency, and the Lustre servers hate that and keep ejecting  
the client.  We haven't solved the problem yet.

Tim


-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.