Q: 64-bit PCI - 66 Mhz vs 33 Mhz networking performance?

[Don Holmgren]

I hadn't known about an i860 problem until Patrick's mail.  
Unfortunately I can confirm the bad news.  Thanks for pointing this out,
Patrick.

On a Dell Precision 530, gm_debug on a Myrinet PCI64B plugged into the
64/66 PCI bus gives:

  [root at fcwork2 bin]# ./gm_debug 
  Opening board 0, port 0

  DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
  1st: 8 pages from bogus sdma pg, 8 to bogus rdma
          bus_read  (send) = 146 MBytes/s
          bus_write (recv) = 315 MBytes/s

  DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
  2nd: sdma a page, rdma a page 8 times
          bus_read  (send) = 146 MBytes/s
          bus_write (recv) = 315 MBytes/s

  DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
  3rd: sdma and rdma to/from alternating pages (coarse grain)
          bus_read  (send) = 146 MBytes/s
          bus_write (recv) = 292 MBytes/s
 
  DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
  4th: sdma and rdma to/from alternating pages (fine grain)
          bus_read  (send) = 146 MBytes/s
          bus_write (recv) = 292 MBytes/s

So, even though the card is capable of sourcing data at nearly
250MB/sec, on this machine sends will be throttled to less than 150MB/sec. 

Don Holmgren
Fermilab


On Tue, 3 Jul 2001, Patrick Geoffray wrote:

> 
> Don Holmgren wrote:
> 
> > The motherboards recently released for dual Pentium IV Xeons which are
> > based on the i860 chipset have 64/66 PCI slots.  The ones that I know
> > of are the Tyan Thunder-i860 and the SuperMicro P4DC6, as well as the
> > motherboards used in the Dell Precision 530 and Compaq W8000.
> 
> i860 is cheesy. It has the 64/66 label but not the performance.
> 
> There is not a lot of choice for PCI 64/66 :
> * ServerWorks: the best, very good PCI chipset.
> * Intel i860: Don't wonder why Intel is using ServerWorks 
> chipset for high-end servers.
> Yes, the 760MP is 64/33.
> 
> For the future, ServerWorks is working on a P4 chipset and 
> the 760MPX should be 64/66. I expecte very good perf from 
> ServerWorks, but I have no clue about 760MPX PCI 
> implementation.
> 
> For GigE or Giganet, 64/33 is good enough (peak at 250 MB/s, should 
> support full-duplex bandwidth). For SCI or Myrinet, you really want 
> 64/66.
> 
> Regards
> 
> Patrick
> 
> -----------------------------------------------------------
> |   Patrick Geoffray, Ph.D.      patrick at myri.com         |
> |   Myricom, Inc.                http://www.myri.com      |
> |   Cell: 865-389-8852           325 N. Santa Anita Ave.  |
> |   Fax:  865-974-1950           Arcadia, CA 91006        |
> -----------------------------------------------------------