[Beowulf] NFS + IB?
laytonjb at gmail.com
Sun Feb 22 10:09:40 PST 2015
Dell has been doing some really good things around NFS and IB using IPoIB.
Their NSS storage solution is an example. They've gotten some pretty good
sequential performance: 2.25 GB/s for writes and 5.48 GB/s for reads.
Pretty good performance from a simple NFS server using IPoIB I think.
They publish the settings they use and also the tests they run. For more
information just Google for Dell NFS, NSS. There should be pdf's that are
The last results I can find are for NSS v5.5 which uses RHEL 6.4. No newer
kernels I'm afraid.
On Fri, Feb 20, 2015 at 7:15 PM, Bill Broadley <bill at cse.ucdavis.edu> wrote:
> I read through the beowulf archives for mentions of NFS + IB. I found
> newer than 2012.
> What are peoples current experience with NFS + IB? I'm looking at the
> for smaller clusters with /home on NFS. I'll leave distributed
> filesystems for
> a separate discussion.
> The two leading options seem to be IPoIB and RDMA. Any comments on
> and performance assuming a fairly recent kernel like 3.13 or newer?
> I'd be especially interested in any particular settings like rsize, wsize,
> (2044 for datagram mode), connected mode vs datagram mode, and similar
> parameters. If you tuned for random I/O or bandwidth please mention it.
> I talked to one vendor that claims FDR has a larger than expected
> advantage over QDR for NFS. Can anyone verify or guess as to exactly why?
> Not sure if the IPoIB tuning is relevant for NFS + RDMA.
> I've read these links, if there are other useful links please let me know:
> I'm happy to summarise the results if people prefer sending info directly
> to me.
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Beowulf