[Beowulf] Cheap SDR IB
hahn at mcmaster.ca
Wed Jan 30 05:58:18 PST 2008
> Just in case you've missed the announcements:
I'm always happy about new levels pricing agression, but I'm
a bit puzzled about for what kind of workloads this will matter.
whenever I ask about IB bandwidth, people always point fingers
at weather codes, which apparently are fond of doing the transpose
in multi-dimension FFT's using all-to-all. while convenient, this
seems a bit silly, since transpose is O(N) communications, not O(N^2).
higher bandwidth/node also makes sense if you're configuring fairly
fat nodes (many cores, probably also lots of ram). but if you do that,
you also amortize the networking, so a cheaper IB setup matters less.
perhaps there are some extremely file-IO intensive workloads that
can sustain ~1 GB/s, but I'd expect them to require some hefty
fileserving hardware, which would also hide the IB cost.
IB for gaming? I have one ratio: 1e-1/3e-6. that's human reaction
time versus IB latency.
also, I think it's a bit disingenous to use 10G Chelsio TOE to compare,
rather than 10G Myri which is cheaper and faster. also:
finally, how the heck do you make Gb as slow as 120 us?
-Mark "not actually anti-IB" Hahn.
PS: does anyone have first-hand experience with ConnectX performance?
More information about the Beowulf