malloc and 'Active' memory

[Deron Meranda]

> A task that has been running for some time may (will!) have fragemented
> Virtual Address space.  A new task should not (though it is possible, say,
> if a shared library has been set to load at a poorly chosen fixed address).

This type of fragmentation is not caused by malloc.  It is most typically
the result of the dynamic library loader, which does not use malloc,
but instead uses mmap directly.  Usually once the program has been
fully loaded and initialized this memory will not change as the program
continues to run, and likewise will be released (to that process) when it
terminates.

However your observation is correct that a fixed-address library may
subdivide the available virtual memory address into smaller regions.
If you're latter trying to malloc a very large segment that's close
the the virtual memory size (depends on 32 or 64 bit), then you can
run into problems.

You can get a hint as to where a process is using memory by
looking at /proc/12345/maps (where 12345 is the process id of your
mysqld process).  It should be easy to see if a library is responsible
for splitting your memory into too-small chunks.  Also the file
/proc/12345/status might have useful information.

Check out the man page for ld.so(8), as there are environment
variables you can set to get debugging information out of the
process loading stage.

Also, if you've disabled your overcommit per above, then make sure
you have enough paging/swapping space available or that can
also cause memory allocations to fail.  It should be at least as large
as the amount of physical memory.  cat /proc/swaps

Although I don't think mysql uses it, SystemV style shared memory
is not usually released when a program terminates, until it's
released manually.  Run "ipcs -a" as root to see what's there.

> When a task exits, all its (non-shared) memory is returned to the OS.
> Shared memory is returned to the OS when the last sharing task exits.

Except for SystemV style shared memory which is not released.

Also, when memory that once held shared libraries is freed, it is
made available for re-use, but it's not immediately flushed, under
the anticipation that the same library will be reloaded soon.
Note that under some other OSs, like AIX, shared libraries once
loaded tend to stay in memory and it's resources unavailable to
other things.  Linux is more forgiving.

Deron Meranda