[Beowulf] mem consumption strategy for HPC apps?

[Bill Rankin]

Mark Hahn wrote:
>
>>Knowing this we could say that HPC applications generally want to eat 
>>all available memory but not more. All available memory here means all 
> 
> 
> I don't believe this is a useful generalization.  applications have a 
> "natural" size.  applications which are blindly scaled up in data
> (without corresponding scaling of NCPUS) are highly suspect, IMO.

In general I see your point (and to some extent agree with it) but there 
are always some exceptions.  Some algorithms (multipole N-body for 
instance) have tunable parameters that allow you to trade off things 
like accuracy for memory footprint.  This in turn affects other 
computational aspects of the process, ad nauseum.  Understanding how 
these tradeoffs interact and can impact the overall performance can be 
quite a challenge and often beyond the abilities of many users, but they 
exist none the less.

More memory may also allow you to run on fewer processors which may well 
enable you to operate within a "sweet spot" on the scalability chart for 
distributed apps.


-bill