Q: 64-bit PCI - 66 Mhz vs 33 Mhz networking performance?

[Jared Hodge]

Patrick,
	How does this translate into actual performance with Myrinet?  I'm
assuming it might affect bandwidth, but not latency.  How much is this
affect in comparison to the supermicro LE chipset?  Thanks.

Jared

Patrick Geoffray wrote:
> 
> Hi Wes,
> 
> W Bauske wrote:
> 
> > That's interesting. Do you by chance have numbers for ServerWorks
> > and say, a 32bit PCI config? Just curious how bad 150MB/sec is.
> > For what I do, that would be more than enough.
> 
> You need to keep in mind the (theor.) peak of the PCI bus :
> * 32 bits/33 Mhz : aggregated ideal peak at 132 MB/s
> * 64 bits/33 Mhz : aggregated ideal peak at 264 MB/s
> * 64 bits/66 Mhz : aggregated ideal peak at 528 MB/s
> 
> On a Intel 440GX 32/33 :
> ================================
> 
> DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
> 1st: 8 pages from bogus sdma pg, 8 to bogus rdma
>         bus_read  (send) = 128 MBytes/s
>         bus_write (recv) = 128 MBytes/s
> 
> DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
> 2nd: sdma a page, rdma a page 8 times
>         bus_read  (send) = 128 MBytes/s
>         bus_write (recv) = 128 MBytes/s
> 
> DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
> 3rd: sdma and rdma to/from alternating pages (coarse grain)
>         bus_read  (send) = 128 MBytes/s
>         bus_write (recv) = 128 MBytes/s
> 
> DMA rate for 4096 Byte transfers (32bit / 33MHz bus)
> 4th: sdma and rdma to/from alternating pages (fine grain)
>         bus_read  (send) = 128 MBytes/s
>         bus_write (recv) = 128 MBytes/s
> 
> => that's good (Intel chipsets are not bad for 32/33)
> 
> On ServerWorks 64/66 (LE):
> ==========================
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 1st: 8 pages from bogus sdma pg, 8 to bogus rdma
>         bus_read  (send) = 455 MBytes/s
>         bus_write (recv) = 512 MBytes/s
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 2nd: sdma a page, rdma a page 8 times
>         bus_read  (send) = 455 MBytes/s
>         bus_write (recv) = 512 MBytes/s
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 3rd: sdma and rdma to/from alternating pages (coarse grain)
>         bus_read  (send) = 455 MBytes/s
>         bus_write (recv) = 512 MBytes/s
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 4th: sdma and rdma to/from alternating pages (fine grain)
>         bus_read  (send) = 455 MBytes/s
>         bus_write (recv) = 512 MBytes/s
> 
> => Well, compare it to the i860. But be carefull, some ServerWorks
> chipsets are not so good.
> For example the HE-LE (64/66) :
> ================================
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 1st: 8 pages from bogus sdma pg, 8 to bogus rdma
>         bus_read  (send) = 315 MBytes/s
>         bus_write (recv) = 372 MBytes/s
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 2nd: sdma a page, rdma a page 8 times
>         bus_read  (send) = 315 MBytes/s
>         bus_write (recv) = 372 MBytes/s
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 3rd: sdma and rdma to/from alternating pages (coarse grain)
>         bus_read  (send) = 315 MBytes/s
>         bus_write (recv) = 372 MBytes/s
> 
> DMA rate for 4096 Byte transfers (64bit / 66MHz bus)
> 4th: sdma and rdma to/from alternating pages (fine grain)
>         bus_read  (send) = 315 MBytes/s
>         bus_write (recv) = 372 MBytes/s
> 
> The best way to choose a machine is to bench the PCI bus before
> to buy, ask from a loan from the vendor.
> 
> A good idea would be to collect PCI performances from different
> motherboards (and different chipsets) and put them on a website
> to help people to make their choice. If someone has some free
> time...
> 
> -----------------------------------------------------------
> |   Patrick Geoffray, Ph.D.      patrick at myri.com         |
> |   Myricom, Inc.                http://www.myri.com      |
> |   Cell: 865-389-8852           325 N. Santa Anita Ave.  |
> |   Fax:  865-974-1950           Arcadia, CA 91006        |
> -----------------------------------------------------------
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Jared Hodge
Institute for Advanced Technology
The University of Texas at Austin
3925 W. Braker Lane, Suite 400
Austin, Texas 78759

Phone: 512-232-4460
Fax: 512-471-9096
Email: Jared_Hodge at iat.utexas.edu