[Beowulf] NFS over RDMA

[Joe Landman]

John Hearns wrote:
> Joe has just mentioned NFS over RDMA, a subject which I have somewhat of 
> a passing interest in.
> What's the perceived wisdom on it? Biggest, best shiny new thing on the 
> block? Works just fine?
> Or should it be in the basement inside a locked filing cabinet in a 
> disused lavatory with a sign saying Beware of the Leopard?

FWIW:  we see ~460 MB/s over SDR pairs, going disk to disk (that is real 
disk on JackRabbit behind the RDMA as opposed to ram disk or null 
devices which give obscenely fast, and practically unattainable numbers 
in real world scenarios).

We have wedged one machine running it, requiring a LART (reset button 
press) on the client side.  On the server side, it still is finicky but 
this is mostly on initial mount.

Requires mount-utils of 1.1.1 or higher (1.1.2 or higher is 
recommended).  Requires kernels of 2.6.25 or higher on both sides due to 
in kernel support.  Not sure if the OFED stack does some sort of magic 
here for people.



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
        http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615