[Beowulf] Cheap SDR IB

[Gilad Shainer]

> 
> I thought the thrust of the original post was that you can 
> build now a cheap IB cluster with up to 24 nodes. The 
> subsequent discussion was around questioning whether you need 
> IB for up to 16-24 nodes. 
> The advantage you point to is for 32 nodes. There is no 
> question that IB is much better at this scaling point for 
> many codes, not just WRF.
> I used to think that the typical break point is about 16 
> nodes. We have a lot of app data on our web site which 
> confirm this and will be looking into how (if at all) quad 
> cores change this.
> 


With more cores on a single node, the IB benefits are seen in much lower number of nodes. I am testing some applications on a new cluster that I have (dual sockets quad core Barcelona), and my first results are with Fluent new benchmarks. I will have the numbers posted soon, so you all can take a look. For 2 nodes, IB shows an average of 15-20% higher performance then GigE, and this gap gets bigger with cluster size. At 4 nodes the difference was 40-50%. Even more important, 3 nodes results with IB were higher then 8 nodes with GigE, and GigE stop scaling after 3-4 nodes (performance numbers were flat after 3-4 nodes). 

This is only one example, I know, but I am sure there will be many more. I personally going to check it on more applications, and would appreciate any suggestion on other applications people have interest to check.



> As far as the games are concerned never underestimate what 
> people can do if sufficient and affordable resources are 
> provided. If the opposite were true we'd be still staring at 
> black and white/green screens and playing packman. Game 
> developers have to offload a lot (too much in fact) to a 
> client out of necessity, not because they want to (BTW, there 
> are amazing exploits because of the lags and the way the 
> clients are built). Having said that there is no question 
> that IB is hardly applicable to MMORPGs on the scale they 
> are, probably not even for LAN parties. However a small 
> cluster for real time ray tracing might be a good proposition.
> 
> Igor
>