Cluster and RAID 5 Array bottleneck.( I believe)
lindahl at conservativecomputer.com
Mon Mar 19 11:19:53 PST 2001
On Thu, Mar 15, 2001 at 11:07:43AM -0600, Leonardo Magallon wrote:
> A colleague of mine is saying that the problem is at the network
> level and I am thinking that it is at the Array level because the
> lights on the array just keep steadily on and the switch is not even
> at 25% utilization and attaching a console to the array is mainly
> for setting up drives and not for monitoring.
One way to approach this problem is to test the two independently.
To test the array, you can dd a large file on the server. It needs to
be big enough that you don't get any caching effects. Something like:
dd if=/dev/zero of=/array/big.file bs=1024k count=2048 (2 gigabytes)
time dd if=/array/big.file bs=1024k
But, to simulate your client's workload better, you would want to
start one smaller dd for each client. Reading N independent files is
harder for an array than 1 big file.
To test the network, create a small file that does fit into memory,
and rcp it to all the nodes. That doesn't have the inefficiency of NFS
(especially UDP) but it gives you a "speed of light" for the network:
dd if=/dev/zero of=/tmp/delete.me bs=1024k count=50 (50 megabytes)
rcp /tmp/delete.me client001:/tmp/delete.me
Again you need one rcp per client, simultaneously.
To test with NFS, read the same small file simultaneously from every
time cp /array/50.megabyte.file /tmp/delete.me
More information about the Beowulf