[Beowulf] many cores and ib

[Jan Heichler]

Hallo Jaime,

Montag, 5. Mai 2008, meintest Du:

JP> Hello,

JP> Just a small question, does anybody has experience with many core
JP> (16) nodes and infiniband? Since we have some users that need
JP> shared memory but also we want to build a normal cluster for
JP> mpi apps, we think that this could be a solution. Let's say about
JP> 8 machines (96 processors)  pus infiniband. Does it sound correct?
JP> I'm aware of the bottleneck that means having one ib interface for 
JP> the mpi cores, is there any possibility of bonding?

i had another idea. Roughly one can say that a quad socket system is three times as expensive as a dual socket system -> so you spend a lot of money for "some users". Maybe it is worth thinking about using dual-socket machines and test if scaleMP gives you a good performance over two of them combined. I have no experience with that but maybe you can find that out...

For the interconnect it would be useful to know what and how your application is communicating. Big messages? Small messages? Complex operations? Maybe you can find out.

Cheers,
Jan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.scyld.com/pipermail/beowulf/attachments/20080506/d84e357a/attachment.html