PCI-64: how to find

[Velocet]

On Mon, Feb 25, 2002 at 03:22:20PM -0500, Patrick Geoffray's all...

> >>Euh, yes, I think they are: 6.5 us with a 64/66 PCI, 8 us with a 32/33 
> >>PCI (Lanai9 at 200 MHz). The main difference here is the PCI clock, the 
> >>latency with a 64/33 MHz is between 7.5 and 8 us.
> >>
> > 
> > with what size packets? 
> 
> 0-sized packets.
> 
> > Wouldnt the latency reduce for a several hundred
> > byte packet just because the bus is that much wider? and twice the effect
> > for 66/64 pci? 
> 
> A 64 bits PCI is not always 2 times faster than a 32 bits one. It's true 
> for medium and large transactions, but not for tiny packets where the 
> number of cycles used for the arbitration and to reach the PCI device is 
> the main parameter. In short, 64 bits is good to do burst, but 66 Mhz is 
> what you want to just cross the PCI.

right. I dont know if using LAM/MPI ends up generating the same sized
packets, but for a number of the jobs I've run with gromacs under it
they all seem to be around 700-1000 bytes. Is that big enough to get a win
out of 64 bit busses?

> 
> > Is the cost of 64 gigabit cards vs 32 worth it for things
> > sensitive to latency?
> 
> I would say there should not be any difference in cost between 64 bits 
> and 32 bits cards these days. If we are talking about Gigabit Ethernet, 
> there would be no latency difference for tiny packets, it's so small 
> compared to the IP stack latency.

The cost differences are almost 2x the price for super cheap cards. If you
go with quality (intel, higher end Dlinks) its not as big a difference.
(ie ARKs being $38 for 32bit and $66 for 64bit for low end stuff).

How much latency is due to the stack then? Can this be improved?

/kc
 
> Patrick
> 
> ----------------------------------------------------------
> |   Patrick Geoffray, Ph.D.      patrick at myri.com
> |   Myricom, Inc.                http://www.myri.com
> |   Cell:  865-389-8852          685 Emory Valley Rd (B)
> |   Phone: 865-425-0978          Oak Ridge, TN 37830
> ----------------------------------------------------------
> 

-- 
Ken Chase, math at velocet.ca  *  Velocet Communications Inc.  *  Toronto, CANADA