VM and performance (was Re: [Beowulf] best Linux distribution)

[Bill Rankin]

On Oct 9, 2007, at 9:51 AM, Joe Landman wrote:

> Jeffrey B. Layton wrote:
>> The recent emails from rgb and Doug lead me to a question. Has anyone
>> tested codes running under a VM versus running them "natively" on
>> the hardware (native isn't a good word and I hope everyone gets my
>> meaning)?  The last word I heard is that performance takes a  
>> substantial
>
> You have two major types, the heavyweight emulators (VMware, et al)  
> and the lighter weight hypervisors.  I have seen studies of various  
> codes that show not a huge hit ... the caveat being that these  
> codes spent very little time in system calls (usually thunked to  
> the host somehow) and most of the time computing.


Yup.  My experience (although a little dated) was looking a codes  
running under Xen (see previous post by RGB regarding the COD project  
here at Duke).  Computationally wise, there can be very low impact if  
the system is set up right.  With earlier versions of Xen, there was  
significant overhead in the networking layer which could impact MPI  
and other distributed codes that utilize the IP stack.  This was  
supposedly vastly improved in later versions of Xen, but I don't have  
any data at hand.

I'm not sure how the user-space communication fabrics (OpenIB, etc.)  
would fair under a VM environment, but theoretically they could be  
quite good.  It could make the VM overhead much more palatable in an  
HPC environment.

-bill