Intel 860 PCI bandwidth problem

[Maurice Hilarius]

In recent test on motherboards with Intel 860 chipsets we were seeing less 
than wonderful transfer rates using Wulfkit and Myrinet cards.

After some explorations on kernel issues, and other hardware forums we were 
still not seeing any reason why this was happening.

Recently Intel published updated chipset errata lists, and I scanned over them.

One issue quickly popped out at me, and I now know what the problem seems 
to be:
In the file found at:
ftp://download.intel.com/design/chipsets/specupdt/29071501.pdf

Intel lists errata for the 860 chipset.
One of these states:
"5. Sustained PCI Bandwidth Problem:
  During a memory read multiple operation, a PCI master will read more than 
one complete cache line from memory. In this situation, the MCH pre-fetches 
information from memory to provide optimal performance. However, the MCH 
cannot provide information to the PCI master fast enough. Therefore, the 
ICH2 terminates the read cycle early to free up the PCI bus for other PCI 
masters to claim.

Implication: The early termination limits the maximum bandwidth to ~90 MB/s.

Workaround: None

Status: Intel has no fix planned for this erratum."

This effectively eliminates the 860 chipset motherboards from contention 
for HPTC clustering use, IMHO.

Any thoughts from anyone on this?

With our best regards,

Maurice W. Hilarius       Telephone: 01-780-456-9771
Hard Data Ltd.               FAX:       01-780-456-9772
11060 - 166 Avenue        mailto:maurice at harddata.com
Edmonton, AB, Canada      http://www.harddata.com/
    T5X 1Y3