[Freeipa-users] 389-ds memory usage

Rich Megginson rmeggins at redhat.com
Mon Apr 23 18:38:38 UTC 2012


On 04/23/2012 12:17 PM, Sigbjorn Lie wrote:
> On 04/23/2012 05:45 PM, Rich Megginson wrote:
>> On 04/23/2012 09:22 AM, Sigbjorn Lie wrote:
>>>> Try increasing your nsslapd-cachememsize and monitoring it closely.
>>>> Using the size of id2entry.db4 is a good place to start, but that will
>>>> not be enough.
>>>>
>>>> http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Monitori 
>>>>
>>>> ng_Server_and_Database_Activity-Monitoring_Database_Activity.html
>>>>
>>>> See also https://fedorahosted.org/389/ticket/51 and
>>>> https://bugzilla.redhat.com/show_bug.cgi?id=697701
>>>>
>>> I'm using the latest available server in RHEL 6, has the fix for 
>>> those bugs been applied to this
>>> version?
>> No.  The fix will be in RHEL 6.3
>
>>> 389-ds-base-1.2.9.14-1.el6_2.2.x86_64
>>>
>>> How much do you recommend increasing this too? My id2entry.db4 file 
>>> is 7.2 MB.
>>
>> Hard to day.  Start out with 14.4 MB and monitor the cache usage over 
>> time.
>>
>>>
>>
>
> Ok, will do. A restart of the directory server seem to be overdue. 
> This was waiting for me today at the same server that I took the 
> screendump of "top" yesterday...
>
> It does take a few weeks to build up, so it might take me some time to 
> respond with any findings.

Ok.  The current theory is that the memory growth is caused by the churn 
of entries being added to and removed from the entry cache.  It's not 
yet known why this growth is seen.  It could be just that the memory is 
getting fragmented, or there is a real yet undetected memory leak.  
That's why entry cache sizing and monitoring is very important, to see 
if you are churning entries in/out of the cache, and if that is 
correlated with the memory growth.

>
>
> named invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, 
> oom_score_adj=0
> named cpuset=/ mems_allowed=0
> Pid: 3997, comm: named Not tainted 2.6.32-220.4.2.el6.x86_64 #1
> Call Trace:
>  [<ffffffff810c2b51>] ? cpuset_print_task_mems_allowed+0x91/0xb0
>  [<ffffffff811138d0>] ? dump_header+0x90/0x1b0
>  [<ffffffff8120d69c>] ? security_real_capable_noaudit+0x3c/0x70
>  [<ffffffff81113d5a>] ? oom_kill_process+0x8a/0x2c0
>  [<ffffffff81113c91>] ? select_bad_process+0xe1/0x120
>  [<ffffffff811141b0>] ? out_of_memory+0x220/0x3c0
>  [<ffffffff81123ece>] ? __alloc_pages_nodemask+0x89e/0x940
>  [<ffffffff81158a1a>] ? alloc_pages_current+0xaa/0x110
>  [<ffffffff81110cf7>] ? __page_cache_alloc+0x87/0x90
>  [<ffffffff8112691b>] ? __do_page_cache_readahead+0xdb/0x210
>  [<ffffffff81126a71>] ? ra_submit+0x21/0x30
>  [<ffffffff81111fc3>] ? filemap_fault+0x4c3/0x500
>  [<ffffffff8113b164>] ? __do_fault+0x54/0x510
>  [<ffffffff8113b717>] ? handle_pte_fault+0xf7/0xb50
>  [<ffffffff8141bee1>] ? sock_aio_read+0x181/0x190
>  [<ffffffff8113c354>] ? handle_mm_fault+0x1e4/0x2b0
>  [<ffffffff811762ba>] ? do_sync_read+0xfa/0x140
>  [<ffffffff81042b79>] ? __do_page_fault+0x139/0x480
>  [<ffffffff81218cef>] ? selinux_file_permission+0xbf/0x150
>  [<ffffffff8103758c>] ? kvm_clock_read+0x1c/0x20
>  [<ffffffff814f23fe>] ? do_page_fault+0x3e/0xa0
>  [<ffffffff814ef7b5>] ? page_fault+0x25/0x30
> Mem-Info:
> Node 0 DMA per-cpu:
> CPU    0: hi:    0, btch:   1 usd:   0
> Node 0 DMA32 per-cpu:
> CPU    0: hi:  186, btch:  31 usd:  67
> active_anon:347835 inactive_anon:119039 isolated_anon:0
>  active_file:49 inactive_file:3712 isolated_file:0
>  unevictable:0 dirty:1 writeback:0 unstable:0
>  free:13252 slab_reclaimable:2830 slab_unreclaimable:13380
>  mapped:827 shmem:59 pagetables:4953 bounce:0
> Node 0 DMA free:8356kB min:332kB low:412kB high:496kB 
> active_anon:2952kB inactive_anon:3832kB active_file:116kB 
> inactive_file:232kB unevictable:0kB isolated(anon):0kB 
> isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB 
> mapped:120kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:4kB 
> kernel_stack:8kB pagetables:44kB unstable:0kB bounce:0kB 
> writeback_tmp:0kB pages_scanned:160 all_unreclaimable? no
> lowmem_reserve[]: 0 2004 2004 2004
> Node 0 DMA32 free:44652kB min:44720kB low:55900kB high:67080kB 
> active_anon:1388388kB inactive_anon:472324kB active_file:80kB 
> inactive_file:14616kB unevictable:0kB isolated(anon):0kB 
> isolated(file):0kB present:2052308kB mlocked:0kB dirty:8kB 
> writeback:0kB mapped:3188kB shmem:236kB slab_reclaimable:11320kB 
> slab_unreclaimable:53516kB kernel_stack:2320kB pagetables:19768kB 
> unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:277 
> all_unreclaimable? yes
> lowmem_reserve[]: 0 0 0 0
> Node 0 DMA: 1*4kB 2*8kB 3*16kB 3*32kB 4*64kB 2*128kB 2*256kB 2*512kB 
> 0*1024kB 1*2048kB 1*4096kB = 8356kB
> Node 0 DMA32: 917*4kB 677*8kB 347*16kB 212*32kB 117*64kB 57*128kB 
> 13*256kB 4*512kB 1*1024kB 1*2048kB 0*4096kB = 44652kB
> 6230 total pagecache pages
> 2406 pages in swap cache
> Swap cache stats: add 288041, delete 285635, find 4565761/4568506
> Free swap  = 0kB
> Total swap = 1048568kB
> 524284 pages RAM
> 10513 pages reserved
> 6827 pages shared
> 494838 pages non-shared
> [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
> [  493]     0   493     2782       57   0     -17         -1000 udevd
> [ 1187]     0  1187     1553        2   0       0             0 
> portreserve
> [ 1194]     0  1194    62187       93   0       0             0 rsyslogd
> [ 1236]    32  1236     4756       44   0       0             0 rpcbind
> [ 1254]    29  1254     5800       20   0       0             0 rpc.statd
> [ 1286]     0  1286     7376       31   0       0             0 
> rpc.idmapd
> [ 1306]     0  1306     9841       44   0       0             0 rpc.gssd
> [ 1347]   496  1347   845694   366306   0       0             0 ns-slapd
> [ 1419]   498  1419   297187    11273   0       0             0 ns-slapd
> [ 1499]     0  1499    30950     7834   0       0             0 krb5kdc
> [ 1511]     0  1511    14022       20   0       0             0 
> ipa_kpasswd
> [ 1640]   497  1640   415291    24719   0       0             0 java
> [ 1691]    81  1691    24853       66   0       0             0 
> dbus-daemon
> [ 1703]     0  1703    47295        2   0       0             0 cupsd
> [ 1728]     0  1728     1033        1   0       0             0 acpid
> [ 1737]    68  1737     6776      169   0       0             0 hald
> [ 1738]     0  1738     4539        2   0       0             0 
> hald-runner
> [ 1784]     0  1784     5068        2   0       0             0 
> hald-addon-inpu
> [ 1801]    68  1801     4464        2   0       0             0 
> hald-addon-acpi
> [ 1858]     0  1858    44108       98   0       0             0 sssd
> [ 1864]     0  1864    63923      714   0       0             0 sssd_be
> [ 1876]     0  1876    16024       26   0       0             0 sshd
> [ 1884]    38  1884     8069       66   0       0             0 ntpd
> [ 1891]     0  1891    43086      207   0       0             0 sssd_nss
> [ 1892]     0  1892    44935      234   0       0             0 sssd_pam
> [ 1978]     0  1978    19679       26   0       0             0 master
> [ 1985]    89  1985    20260       60   0       0             0 qmgr
> [ 2002]     0  2002    29709       51   0       0             0 abrtd
> [ 2010]     0  2010     2304       19   0       0             0 
> abrt-dump-oops
> [ 2024]     0  2024    27120      114   0       0             0 ksmtuned
> [ 2033]     0  2033    29312       25   0       0             0 crond
> [ 2044]     0  2044     5373        5   0       0             0 atd
> [ 2062]     0  2062    14288       27   0       0             0 
> certmonger
> [ 2231]     0  2231     7556       19   0       0             0 cfservd
> [ 2378]     0  2378    24457      131   0       0             0 cfexecd
> [ 2452]     0  2452     7640      149   0       0             0 cfenvd
> [ 2553]     0  2553     1029        2   0       0             0 mingetty
> [ 2555]     0  2555     1029        2   0       0             0 mingetty
> [ 2557]     0  2557     1029        2   0       0             0 mingetty
> [ 2559]     0  2559     1029        2   0       0             0 mingetty
> [ 2561]     0  2561     1029        2   0       0             0 mingetty
> [ 2563]     0  2563     1029        2   0       0             0 mingetty
> [ 2569]     0  2569     3111      108   0     -17         -1000 udevd
> [ 2587]     0  2587    23312       41   0     -17         -1000 auditd
> [ 2923]     0  2923   187259      776   0       0             0 automount
> [ 3999]     0  3999     9846       40   0       0             0 
> nss_pcache
> [ 4001]     0  4001   221259     1345   0       0             0 httpd
> [ 3996]    25  3996    56544    12252   0       0             0 named
> [30951]     0 30951     6053       59   0       0             0 xinetd
> [31131]     0 31131     2781       54   0     -17         -1000 udevd
> [30457]     0 30457    25642      407   0       0             0 sshd
> [30464]  5000 30464    25642      406   0       0             0 sshd
> [30465]  5000 30465    27640      116   0       0             0 bash
> [28720]     0 28720    25642      406   0       0             0 sshd
> [28723]  5000 28723    25642      405   0       0             0 sshd
> [28724]  5000 28724    27615      109   0       0             0 bash
> [31792]    48 31792   263393    10920   0       0             0 httpd
> [31793]    48 31793   263393    10920   0       0             0 httpd
> [31794]    48 31794   227580     2198   0       0             0 httpd
> [31795]    48 31795   227580     2198   0       0             0 httpd
> [31796]    48 31796   227580     2198   0       0             0 httpd
> [31797]    48 31797   227580     2199   0       0             0 httpd
> [31798]    48 31798   227580     2200   0       0             0 httpd
> [31799]    48 31799   227580     2204   0       0             0 httpd
> [31800]    48 31800   227580     2198   0       0             0 httpd
> [31801]    48 31801   227580     2198   0       0             0 httpd
> [23118]    48 23118   227580     2198   0       0             0 httpd
> [14842]    89 14842    20216      204   0       0             0 pickup
> [16012]     0 16012     4334       50   0       0             0 anacron
> [16688]     0 16688    25240       18   0       0             0 sleep
> Out of memory: Kill process 1347 (ns-slapd) score 788 or sacrifice child
> Killed process 1347, UID 496, (ns-slapd) total-vm:3382776kB, 
> anon-rss:1463896kB, file-rss:1328kB
>
>
>




More information about the Freeipa-users mailing list