malloc and 'Active' memory

[Matthijs van der Klip]

On Mon, 1 Aug 2005, Deron Meranda wrote:
> The malloc(3) function is entirely userspace code, part of glibc.  It's
> not a kernel feature.

Yes, I'm aware of that, but I didn't mention for simplicity's sake... ;-)


> The kernel usually allocates memory to processes via the sbrk(2) system
> call in whole pages.  From a user-process perspective this can not be
> fragmented since the pages are virtually mapped into a linear address
> space regardless of their actual physical address.  The malloc(3)
> function though will use mmap(2) instead of sbrk(2) when allocating
> very large chunks of memory; although it's still only getting whole
> pages at a time.
> 
> It is true though that the malloc(3) function in glibc is very advanced and
> is mostly imune to many forms of traditional memory fragmentation.
> For an overview of the algorithms it uses read this document by
> Doug Lea: http://gee.cs.oswego.edu/dl/html/malloc.html
> 
> It's not exactly the same as the Linux implmentation, but it's pretty
> close.  BTW the glibc version comes from ptmalloc2 by Wolfram Gloger,
> which is found here:  http://www.malloc.de/en/index.html

Actually it's not really the malloc behaviour I don't understand, it's the 
definition of 'Active' memory. What is it's use and why is it there? When 
doing a couple of rather heavy compiles the amount of Active memory keeps 
rising until malloc's requesting for large amounts of memory begin to 
fail. The memory is there, but it's being marked as Active which prevents 
it from being allocated all at once. Why is that?

This is the mmap call from a strace session by the way:

mmap(NULL, 7340052480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 
-1, 0) = -1 ENOMEM (Cannot allocate memory)

The default setting for overcommit_memory is set to 0 which means:

0       -       Heuristic overcommit handling. Obvious overcommits of
                address space are refused. Used for a typical system. It
                ensures a seriously wild allocation fails while allowing
                overcommit to reduce swap usage.  root is allowed to 
                allocate slighly more memory in this mode. This is the 
                default.

Now, with a setting of 1 (heuristics turned off) the malloc call obviously 
succeeds as it means:

1       -       Always overcommit. Appropriate for some scientific
                applications.

In other words I think the heuristics are causing my grief. It is just not
making sense to me that allocating 2GB with malloc a hundred times (total
of 200GB) works when overcommit_memory is set to 0, but a single malloc
for 7GB fails. Now which of the two is the obvious overcommitment
situation?

What I've done at the moment is the following:

A) I have set overcommit_memory to 2 (do not overcommit) and 
   overcommit_ratio to 100. This ensures the VM will never commit to more 
   than physical RAM plus swap (I have configured a very small swap 
   partition).

B) I configured MySQL to 4GB innodb_buffer_pool_size.

C) I have set max_connections to 256, which results in a maximum useage of 
   key_buffer_size+(read_buffer_size+sort_buffer_size)*max_connections 
   ~ 2GB of memory for the connecting threads.


Now after starting MySQL I get values as follows from /proc/meminfo:

CommitLimit:   8646184 kB
Committed_AS:  5283720 kB

Now, this leaves 8646184 kB - 5297108 kB ~ 3GB of memory to commit, 2GB 
will be used when max_connections is reached, leaving another 1GB for 
safety purposes.

This configuration is far more on the safe side than I'm used to, but I
think it makes kind of sense, don't you?

Interestingly enough starting MySQL in this situation is no problem, even 
with enormous amounts of 'Active' memory laying around.

Before starting MySQL:

MemTotal:      8124080 kB
MemFree:       1230808 kB
Active:        4098516 kB
Inactive:      2554808 kB
CommitLimit:   8646184 kB
Committed_AS:   240412 kB

After:

MemTotal:      8124080 kB
MemFree:        804764 kB
Active:        4515396 kB
Inactive:      2562880 kB
CommitLimit:   8646184 kB
Committed_AS:  5283720 kB


> Oh, from the malloc.c source code under the function free() we
> have this comment:
> 
>  * Unless disabled (using mallopt), freeing very large spaces will
>  * when possible, automatically trigger operations that give
>  * back unused memory to the system, thus reducing program footprint.
> 
> You can fine-tune the agressiveness when memory is released back
> to the system by using the mallopt(3) call with the option of
> M_TRIM_THRESHOLD.  Or you can manually force a release using
> the malloc_trim(3) function.  (only the non-fragmented memory is
> released).

To my understanding, calling malloc_trim would only be of benefit when a 
process releases large amounts of memory but _continues_ to run, so that 
glibc can return the memory to the OS immediately instead of waiting until 
process termination. Not so?


Regards,

-- 
Matthijs van der Klip
System Administrator
Spill E-Projects
The Netherlands