[Beowulf] What are people seeing performance-wise for NFS over 10GbE

[Jan Heichler]

Hallo Joe,

Donnerstag, 28. Februar 2008, meintest Du:

JL>    Have a few simple setups that I am trying to figure out if I have a
JL> problem, or if what I am seeing is normal.

JL>    Two 10 GbE cards, connected with a CX4 cable.  Server is one of our
JL> JackRabbits, with 750+ MB/s direct IO and 500-650 MB/s buffered IO 
JL> (read-write), for IO about 10x system ram (~100x RAID cache).

JL>    Getting ok iperf numbers, about 7 Gb/s single thread.  Running NFS,
JL> and seeing ~200-300 MB/s best case between the systems.  Is this what 
JL> others have seen?  Latest drivers from vendor, sent a note to them to 
JL> see if we can figure this out.  Figured I would tap into the 'wulf 
JL> collective memory to learn what other people see.

The best i saw for NFS over 10 GE was about 350-400 MB/s write and about 450 MB/s read.
Single server to 8 simultaneous accessing clients (aggregated performance). 

On the blockdevice i got 550 MB/s write and 1.1 GB/s read performance. 


JL>    2.6.23.14 kernel on both sides, jumbo frames enabled. No switch, just
JL> a CX4 cable.

rsize/wsize are set to? 

NFS3 or NFS4? 

Cheers,
Jan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.scyld.com/pipermail/beowulf/attachments/20080228/29412ece/attachment.html