[Beowulf] Virtualisation and high performance interconnects.

[Gilad Shainer]

> On Thu, 1 Nov 2007, andrew holway wrote:
> 
> > I'm trying to find out about the effects of virtualisation on high 
> > performance interconnects. Effects on latency and bandwidth.
> 
> Google is your friend.. :-)
> 
> There is an IBM presentation from the 2006 Xen conference on 
> virtualising InfiniBand networks, including measurements and 
> comparisons with native results.
> 
> http://www.xen.org/files/xs0106_virtualizing_infiniband.pdf
> 
> VMware results might be a bit harder to find given their 
> license restrictions on benchmarking.
> 

When you have a VMM between the virtual machine and physical
interconnect, there are more layers of SW and therefore decrease in IO
performance. This is regardless if you are using OS bypass or not, RDMA
or send/receive etc. Good way to solve this is to use IO that offloads
the VMM, and that move back the "network" jobs from the VMM back to the
network. IO channels with ability to do the protection as part of the
adapter, eliminate the overhead that the VMM creates and get you the
same performance of a native OS. 

Gilad.