Q: 64-bit PCI - 66 Mhz vs 33 Mhz networking performance?

[Patrick Geoffray]

Hi Wes,

W Bauske wrote:

> That's interesting. Do you by chance have numbers for ServerWorks
> and say, a 32bit PCI config? Just curious how bad 150MB/sec is.
> For what I do, that would be more than enough.

You need to keep in mind the (theor.) peak of the PCI bus :
* 32 bits/33 Mhz : aggregated ideal peak at 132 MB/s
* 64 bits/33 Mhz : aggregated ideal peak at 264 MB/s
* 64 bits/66 Mhz : aggregated ideal peak at 528 MB/s

On a Intel 440GX 32/33 :
================================

DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
1st: 8 pages from bogus sdma pg, 8 to bogus rdma
        bus_read  (send) = 128 MBytes/s
        bus_write (recv) = 128 MBytes/s

DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
2nd: sdma a page, rdma a page 8 times
        bus_read  (send) = 128 MBytes/s
        bus_write (recv) = 128 MBytes/s

DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
3rd: sdma and rdma to/from alternating pages (coarse grain)
        bus_read  (send) = 128 MBytes/s
        bus_write (recv) = 128 MBytes/s

DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
4th: sdma and rdma to/from alternating pages (fine grain)
        bus_read  (send) = 128 MBytes/s
        bus_write (recv) = 128 MBytes/s

=> that's good (Intel chipsets are not bad for 32/33)


On ServerWorks 64/66 (LE):
==========================

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
1st: 8 pages from bogus sdma pg, 8 to bogus rdma
        bus_read  (send) = 455 MBytes/s
        bus_write (recv) = 512 MBytes/s

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
2nd: sdma a page, rdma a page 8 times
        bus_read  (send) = 455 MBytes/s
        bus_write (recv) = 512 MBytes/s

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
3rd: sdma and rdma to/from alternating pages (coarse grain)
        bus_read  (send) = 455 MBytes/s
        bus_write (recv) = 512 MBytes/s

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
4th: sdma and rdma to/from alternating pages (fine grain)
        bus_read  (send) = 455 MBytes/s
        bus_write (recv) = 512 MBytes/s

=> Well, compare it to the i860. But be carefull, some ServerWorks 
chipsets are not so good.
For example the HE-LE (64/66) :
================================

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
1st: 8 pages from bogus sdma pg, 8 to bogus rdma
        bus_read  (send) = 315 MBytes/s
        bus_write (recv) = 372 MBytes/s

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
2nd: sdma a page, rdma a page 8 times
        bus_read  (send) = 315 MBytes/s
        bus_write (recv) = 372 MBytes/s

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
3rd: sdma and rdma to/from alternating pages (coarse grain)
        bus_read  (send) = 315 MBytes/s
        bus_write (recv) = 372 MBytes/s

DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
4th: sdma and rdma to/from alternating pages (fine grain)
        bus_read  (send) = 315 MBytes/s
        bus_write (recv) = 372 MBytes/s


The best way to choose a machine is to bench the PCI bus before 
to buy, ask from a loan from the vendor.

A good idea would be to collect PCI performances from different 
motherboards (and different chipsets) and put them on a website 
to help people to make their choice. If someone has some free 
time...

-----------------------------------------------------------
|   Patrick Geoffray, Ph.D.      patrick at myri.com         |
|   Myricom, Inc.                http://www.myri.com      |
|   Cell: 865-389-8852           325 N. Santa Anita Ave.  |
|   Fax:  865-974-1950           Arcadia, CA 91006        |
-----------------------------------------------------------