[Beowulf] 8GB memeory limit?

[Roland Krause]

Martin,
your problem, or observation, is due to a change in the way the kernel
and the glibc facilitate the allocation of memory. 

The background of this is explained very well in this earlier post

http://www.mail-archive.com/beowulf%40beowulf.org/msg03723.html

and that's also where I learned about it, so all I will say here is
that the change that allows you to allocate your entire memory using
mmap occurred in kernel 2.6.9.

Best regards,
Roland




--- Martin Siegert <siegert at sfu.ca> wrote:

> Hi,
> 
> I am running into a bizarre memory issue: I do not appear to be
> able to allocate 8GB of memory into a single array:
> 
> ===<int_malloc.c>================================================
> #include <stdio.h>
> #include <stdlib.h>
> 
> int main(int argc, char *argv[]){
> long int i, n;
> long int *m;
> 
>    n = 1024*1024*1024*sizeof(long int);
>    for (i = 1; i <= n; i*=2) {
>       m = (long *) malloc(i);
>       if (m == NULL) {
>          fprintf(stderr, "allocation of %li bytes failed.\n", i);
>          exit(-1);
>       }
>       free(m);
>    }
> }
> ===</int_malloc>=================================================
> 
> # gcc -m64 int_malloc.c
> # ./a.out
> allocation of 8589934592 bytes failed.
> 
> This is with a 2.6.5 kernel (SLES 9).
> If I compile the same program under kernel 2.6.16.27 (openSuSE 10.2),
> the program completes without problem.
> 
> Under either OS I can allocate, e.g., 5 arrays of 4GB each within the
> same program without problem.
> 
> Where does this limit of 8GB for a single array come from?
> Is it in the kernel? If yes, can it be changed, e.g., through a
> sysctl?
> Which one?
> 
> Cheers,
> Martin
> 
> -- 
> Martin Siegert
> Head, HPC at SFU
> WestGrid Site Lead
> Academic Computing Services                phone: (604) 291-4691
> Simon Fraser University                    fax:   (604) 291-4242
> Burnaby, British Columbia                  email: siegert at sfu.ca
> Canada  V5A 1S6
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>