[Beowulf] Network Filesystems performance

Joe Landman landman at scalableinformatics.com
Thu Aug 23 12:53:51 PDT 2007

Glen Dosey wrote:
> I am using the async flag. I'm also using the Infiniband now instead of
> the GigE and I've increased my default TCP window up to 6MB. None of it
> makes a difference. It's still around 40MB/s.


 dstat, iftop, and atop are your friends.


Nothing like doing an

	dstat -N total,eth0,eth1 -D sda,sdb,...

on both machines before starting the transfer, in different windows of
course ...

	iftop -B -i eth0

is known to be extraordinarily helpful at figuring out what bandwidth
you are really seeing.

Since you indicated RHEL4, its possible that something in kernel is
causing problems.  RHEL4 is not known to be a speed demon.

What about the usual suspects

	cat /proc/interrupts
	blockdev --getra /dev/sda
	lspci -v

Is your gigabit sharing a 100/133 MB/s old PCI bus with your RAID card?
On older motherboards, the gigabit NICs were put on an old PCI branch,
typically 100 MB/s max.  If there is a PCI RAID card in the same slot,
or, as also often happened on these older MB's, the SATA ports were
hanging off the same old/slow PCI bus, well, it could explain your results.

Which MB do you have?  Which bios rev, ...  Which raid card, how much
ram, 32 or 64 bit, yadda yadda yadda (all the details you didnt give


Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 866 888 3112
cell : +1 734 612 4615

More information about the Beowulf mailing list