[Beowulf] Network Filesystems performance

[Glen Dosey]

Reply inline

On Tue, 2007-08-21 at 17:24 +1000, Chris Samuel wrote:
> On Tue, 21 Aug 2007, Glen Dosey wrote:
> 
> > I've left out a lot of detail to keep this succinct. I can provide
> > it when necessary.
> 
> I think the details you've omitted mean there's not enough information 
> there to actually come to any conclusions! :-)
One would might think that ....


> What distro on the clients and servers ?
RHEL5 (and RHEL4 to a lesser extent)


> What does /proc/mounts say on the clients ?
it depends on the test, but the results are almost always the same, I've
played with block sizes up to at least 131078. It is NFS v3 over TCP,
but like I said, SMB and even scp exibit the same 40MB/s performance on
the first read of a file on the server, followed by near line speed
performance on subsequent copies when the file size is less than server
memory size and has been cached. (again note, I have made sure it is not
being cached on the client )


> What is underlying the IP network - 1 gigE, 10 gigE, Infiniband, 
> Myrinet, Quadrics, quantum carrier pigeons 
1GigE and also tried Infiniband, focus on 1GigE


> What is the MTU ?
1500, I am looking into jumbo frame support


> What is the file system type on the NFS server that you export (ext3, 
> XFS, JFS, etc) ?
Ext3, GFS and XFS (where GFS is slightly slower than the numbers I
posted wrt array performance )


> What... is the air-speed velocity of an unladen swallow?  :-)
Do you mean an African or European Swallow? :)


> cheers,
> Chris