unexpected malloc behavior on X86_64 Fedora 7 machines

Richard W.M. Jones rjones at redhat.com
Fri Oct 19 12:49:24 UTC 2007


Flemming Andersen wrote:
> I do not know how wide spread this problem is, but I have not been able 
> to install MoscowML due to an unexpected behavior of the malloc version 
> used by Fedora 7 on my new Intel X86_64 machine. The installation of 
> MoscowML appears to fail due to the malloc behavior as described below.
> 
> I have also found out that mallopt may be a way to control the behavior 
> of malloc, but it is not very well documented in Fedora.
> 
> So I would like to know if mallopt is officially support and if it will 
> continue to be so. Alternatively, I would like to know if you are able 
> to suggest any other solution to the problem described below.
> 
> Description of problem with installing MoscowML under Fedora 7 on X86_64 
> machine:
> ======================================================
> I learned earlier this week from Prof. Peter Sestoft (ITU DK) who owns 
> MoscowML that in my case the problem is caused by certain versions of 
> malloc(), which during the installation of MoscowML alternatively 
> allocate memory for the camlrunm (underlying runtime system for 
> MoscowML) heap in very high addresses using mmap() or in very low 
> addresses using brk().

This is a feature not a bug, and in any case the allocation doesn't 
happen "alternatively", but in response to the size of the block being 
allocated.

Considering that malloc() is probably the most frequently used C library 
function, called by each of the thousands of programs in Fedora with no 
apparent problems, it seems rather less likely that this is a bug in 
glibc, and rather more likely to be a bug or faulty assumption in MoscowML.

 > This span appears to require a huge page_table
> (14 GB or so), which it would be silly to allocate.

By 'page_table' are you referring to processor page table entries, or 
some structure within MoscowML?

 > I also learned from
> Peter that the problem can be avoided by either forcing high memory 
> allocation using this environment variable:
> 
>    export MALLOC_MMAP_THRESHOLD_=0
> 
> or by using forcing low memory allocation using this environment variable:
> 
>    export MALLOC_MMAP_MAX_=0
> 
> However, there may be performance implications of either of these 
> choices. The latter one would limit usable mosml memory to at most 3 GB, 
> I think, whereas the former one has been experimentally tested to allow 
> mosml to use more than 4 GB.
> 
> Also note that these environment variables affect *all* programs that 
> use malloc() and hence may have mysterious side effects.

OCaml uses malloc for underlying allocations and does not experience any 
problems under Fedora 7.

Rich.

-- 
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom.  Registered in
England and Wales under Company Registration No. 03798903
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/fedora-list/attachments/20071019/078dfa17/attachment-0001.bin>


More information about the fedora-list mailing list