[Beowulf] Raid disk read performance issue in Linux 2.6, anyone seen this?

[Craig Tierney]

I have noticed some significant differences in disk performance
when moving some of my cluster I/O servers from linux-2.4 to
linux-2.6.  For several different raid systems with several 
linux-2.6 kernel and distributions, the read performance is
quite poor.  

Distros:  SuSE 9.1 Professional AMD64, Redhat 4.0 beta, IA64
Kernels: Standard distro kernels, also tried stock 2.6.9 on
         Redhat 4/IA64.

For the 2.4 kernels, I needed to tweak /proc/sys/vm/max_readahead
to improve performance.  In 2.6, the command is blockdev --setra.
Using blockdev helps some, but not much.  Adjusting blockdev
gets performance for one raid system from 10 MB/s to 30 MB/s.  Under
2.4 the read performance is 90 MB/s. 

I looked at a few of the anticipatory scheduler options and also
tried using the deadline scheduler.  Nothing helped.

This problem appears on different storage.  I have tried DDN (raid 3),
Nexsan (raid 5), and Infotrend (raid 3 and 5).
  
For those using PVFS1/2, Lustre, or building fast NFS servers
with Linux 2.6, have you seen this?

Thanks,
Craig