Cluster and RAID 5 Array bottleneck.( I believe)

[Greg Lindahl]

On Thu, Mar 15, 2001 at 11:07:43AM -0600, Leonardo Magallon wrote:

> A colleague of mine is saying that the problem is at the network
> level and I am thinking that it is at the Array level because the
> lights on the array just keep steadily on and the switch is not even
> at 25% utilization and attaching a console to the array is mainly
> for setting up drives and not for monitoring.

One way to approach this problem is to test the two independently.

To test the array, you can dd a large file on the server. It needs to
be big enough that you don't get any caching effects. Something like:

dd if=/dev/zero of=/array/big.file bs=1024k count=2048    (2 gigabytes)
time dd if=/array/big.file bs=1024k

But, to simulate your client's workload better, you would want to
start one smaller dd for each client. Reading N independent files is
harder for an array than 1 big file.

To test the network, create a small file that does fit into memory,
and rcp it to all the nodes. That doesn't have the inefficiency of NFS
(especially UDP) but it gives you a "speed of light" for the network:

dd if=/dev/zero of=/tmp/delete.me bs=1024k count=50        (50 megabytes)
rcp /tmp/delete.me client001:/tmp/delete.me

Again you need one rcp per client, simultaneously.

To test with NFS, read the same small file simultaneously from every
client:

rlogin client001
time cp /array/50.megabyte.file /tmp/delete.me

-- greg