[Beowulf] Feedback on large pages in Linux

[Toon Moene]

Valerie Henson wrote:

> Hello all,
> 
> We are doing some work on large page support in Linux.

Yes, Yes, YES !!!
> Please reply directly to me, unless you have something to share with
> the whole list.  Thanks!

Arggghhhh - you can't be serious.

> 1. What does your organization do?

My day time job (i.e., when I'm *not* working on GNU Fortran) at the 
Dutch Weather Service entails improving our Limited Area Numerical 
Weather Forecasting Model - hirlam.org).

> 2. What does your application do?

Forecasts the weather up to 48 hours in advance over the Atlantic + 
Europe (at least, that's the default configuration).

> 3. How are large pages useful to you?

We use lots and lots of memory every time step in the model.  Our 
soon-to-be operational model uses ~ 1 Gbyte of memory per (~100) MPI 
node every time step.  There are 720 time steps (48 hours * 15 time 
steps of 4 minutes real time per hour) during the 40 minute integration. 
  The job maximum time of 40 minutes is determined by our delivery dead 
line.

> 4. What is your experience with the Linux hugetlbfs implementation of
>    large page support?

Haven't tried it yet.  However, our experience on Solaris 9 (which 
supports 4 Mbyte pages down to the compilation/link unit) is very 
positive (this is on UltraSPARC III hardware).

> 5. Does Linux's large page support affect your decision whether or not
>    to use Linux? If so, how?

Nope.  We bought a Linux solution because hw+sw+other stuff was better 
this way (SGI Altix).

> 6. What would make large page support in Linux better for your application?

Duh. Fewer TLB cache reloads.

Cheers,

-- 
Toon Moene - e-mail: toon at moene.indiv.nluug.nl - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
A maintainer of GNU Fortran 95: http://gcc.gnu.org/fortran/
Gfortran At Work: http://mysite.verizon.net/serveall/moene.pdf