[Beowulf] Infiniband modular switches
landman at scalableinformatics.com
Sun Jun 15 09:36:15 PDT 2008
On a different note ...
Gilad Shainer wrote:
> routing capabilities. And if you are bandwidth bounded, using IB QDR
> will help. You will be able to drive more than 3GB/s from each server.
This is theoretical max, correct? What sort of real observed DMA
bandwidth are people seeing with the QDR adapters? That is, how quickly
can I copy 32MB from main memory to the PCIe (even gen2) based card? Or
how quickly (e.g. what is the real observed data rate) for copying
streaming data, say from a nice NFS over RDMA session, to the card?
Please correct me if I am wrong, but I suspect that this will be one of
rate limiting factors at least in the near term.
Not that this is a bad thing, 3GB/s is quite good, but if the practical
best case memory->PCIe bandwidth only let us get 2GB/s, or even 1.5
GB/s, this is important to know.
If you or someone out there with QDR could measure this, I think a few
of us (at least on the storage side) would like to know this. If
someone wants to loan us cards and cables (cough cough), we would be
happy to do this measurement between pairs of JackRabbits.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf