[Beowulf] Help for terrible NFS write performance
landman at scalableinformatics.com
Fri Aug 21 10:38:17 PDT 2009
Orion Poplawski wrote:
> I'm trying to improve the terrible NFS (write in particular) performance
> I'm seeing. Pure network performance does not appear to be an issue as
> I can hit 120MB/s reading which should be about the limit for gigE.
> Perhaps the local disk performance is not what it should be. Any help
> would be greatly appreciated. Using bonnie++ for benchmarks.
> Dual proc dual core opteron 2GHz
> 8GB RAM
> CentOS 4.7
> kernel 2.6.9-78.0.22.plus.c4smp
> 3 8-port Marvell MV88SX6081 SATAII controllers
> sata_mv 3.6.2 driver
> Ethernet controller: nVidia Corporation MCP55 Ethernet (rev a3)
> MTU 8982
> Arrays are linux md arrays of 6 disks with 2 on each controller. 64k
> cunks. ext3 filesystem.
If I had to bet, ext3 would have much to do with this ... though,
honestly, md RAID write performance over NFS is nothing to write home
about. We can get ~350MB/s on our DeltaV's, but this takes lots of work.
> "working" - raid0 ST31000340AS 1TB drives
> local perf: 224-240MB/s write, 135MB/s rewrite, 390-400MB/s read
> "cora6" - raid5 ST31500341AS 1.5TB drives
> local perf: 84MB/s write, 42MB/s rewrite, 161-166MB/s read
> /export *.cora.nwra.com(rw,sync,fsid=0)
> /export/cora6 *.cora.nwra.com(rw,sync,nohide)
> /export/working *.cora.nwra.com(rw,sync,nohide)
Ok. There it is... Sync.
Don't need to see anything else.
That is it.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf