[Fedora-directory-users] notes on building fds in etch and a failed build question
Rich Megginson
rmeggins at redhat.com
Wed Mar 5 15:17:16 UTC 2008
Tamas Bagyal wrote:
> Rich Megginson wrote:
>> Bagyal Tamas wrote:
>>> Rich Megginson wrote:
>>>> Tamas Bagyal wrote:
>>>>> hello Ryan,
>>>>>
>>>>> you tried this version? i have two fedora-ds 1.0.4 in mmr
>>>>> configuration. i migrate one of those to 1.1 (builded by your and
>>>>> Rich's instrutctions). but i have a problem with memory usage of
>>>>> ns-slapd process. initially mem usage is 18.5% but after 2 hours
>>>>> this changed to 23.1% and growed until killed by kernel. (i think...)
>>>>>
>>>>> mostly read transactions happen (dns) with a few write (cups).
>>>>> this is a debian etch, mem size is 512 mbyte (i know this is too
>>>>> low, but this is a test environment). cache size of slapd is
>>>>> 67108864.
>>>> Are you using SSL? Anything interesting in your server error log?
>>>
>>> I running the setupssl2.sh but not use any ssl connection. error log
>>> shows nothing, only the server start.
>> The reason I ask is that older versions of the NSS crypto/SSL
>> libraries had a memory leak. NSS 3.11.7 does not have this problem.
>> But you would only see the problem if you were using SSL connections.
>
> ok. I tried again from begining. fresh install, no ssl, no migration,
> used the setup-ds-admi.pl and setup the mmr with a fedora-ds 1.0.4.
> but nothing changed, memory usage growing...
> All setting is default except the mmr/changelog and access.log is off.
>
> errors:
>
> Fedora-Directory/1.1.0 B2008.059.1017
> tower.fmintra.hu:389 (/opt/dirsrv/etc/dirsrv/slapd-tower)
>
>
> [05/Mar/2008:10:19:20 +0100] - dblayer_instance_start: pagesize: 4096,
> pages: 128798, procpages: 5983
> [05/Mar/2008:10:19:20 +0100] - cache autosizing: import cache: 204800k
> [05/Mar/2008:10:19:21 +0100] - li_import_cache_autosize: 50,
> import_pages: 51200, pagesize: 4096
> [05/Mar/2008:10:19:21 +0100] - WARNING: Import is running with
> nsslapd-db-private-import-mem on; No other process is allowed to
> access the database
> [05/Mar/2008:10:19:21 +0100] - dblayer_instance_start: pagesize: 4096,
> pages: 128798, procpages: 5983
> [05/Mar/2008:10:19:21 +0100] - cache autosizing: import cache: 204800k
> [05/Mar/2008:10:19:21 +0100] - li_import_cache_autosize: 50,
> import_pages: 51200, pagesize: 4096
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Beginning import job...
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Index buffering
> enabled with bucket size 100
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Processing file
> "/tmp/ldifZHth0D.ldif"
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Finished scanning file
> "/tmp/ldifZHth0D.ldif" (9 entries)
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Workers finished;
> cleaning up...
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Workers cleaned up.
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Cleaning up producer
> thread...
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Indexing complete.
> Post-processing...
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Flushing caches...
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Closing files...
> [05/Mar/2008:10:19:21 +0100] - All database threads now stopped
> [05/Mar/2008:10:19:21 +0100] - import userRoot: Import complete.
> Processed 9 entries in 0 seconds. (inf entries/sec)
> [05/Mar/2008:10:19:22 +0100] - Fedora-Directory/1.1.0 B2008.059.1017
> starting up
> [05/Mar/2008:10:19:22 +0100] - I'm resizing my cache now...cache was
> 209715200 and is now 8000000
> [05/Mar/2008:10:19:22 +0100] - slapd started. Listening on All
> Interfaces port 389 for LDAP requests
> [05/Mar/2008:10:22:23 +0100] NSMMReplicationPlugin - changelog program
> - cl5Open: failed to open changelog
> [05/Mar/2008:10:22:24 +0100] NSMMReplicationPlugin - changelog program
> - changelog5_config_add: failed to start changelog
> [05/Mar/2008:10:26:49 +0100] NSMMReplicationPlugin - agmt="cn=replica
> to backup" (backup:389): Replica has a different generation ID than
> the local data.
> [05/Mar/2008:10:32:00 +0100] NSMMReplicationPlugin -
> repl_set_mtn_referrals: could not set referrals for replica
> dc=fmintra,dc=hu: 32
> [05/Mar/2008:10:32:00 +0100] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=fmintra,dc=hu is going
> offline; disabling replication
> [05/Mar/2008:10:32:00 +0100] - WARNING: Import is running with
> nsslapd-db-private-import-mem on; No other process is allowed to
> access the database
> [05/Mar/2008:10:32:13 +0100] - import userRoot: Workers finished;
> cleaning up...
> [05/Mar/2008:10:32:13 +0100] - import userRoot: Workers cleaned up.
> [05/Mar/2008:10:32:13 +0100] - import userRoot: Indexing complete.
> Post-processing...
> [05/Mar/2008:10:32:13 +0100] - import userRoot: Flushing caches...
> [05/Mar/2008:10:32:13 +0100] - import userRoot: Closing files...
> [05/Mar/2008:10:32:14 +0100] - import userRoot: Import complete.
> Processed 12242 entries in 13 seconds. (941.69 entries/sec)
> [05/Mar/2008:10:32:14 +0100] NSMMReplicationPlugin -
> multimaster_be_state_change: replica dc=fmintra,dc=hu is coming
> online; enabling replication
>
> memory usage by top:
>
> top - 10:58:21 up 25 days, 22:36, 2 users, load average: 0.01, 0.13,
> 0.22
> Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie
> Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi,
> 0.0%si, 0.0%st
> Mem: 515192k total, 189600k used, 325592k free, 36472k buffers
> Swap: 489848k total, 18292k used, 471556k free, 106188k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 27647 fds 15 0 464m 47m 25m S 0.0 9.4 1:34.57 ns-slapd
>
>
> top - 11:23:12 up 25 days, 23:01, 2 users, load average: 0.36, 0.27,
> 0.20
> Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie
> Cpu(s): 3.0%us, 0.0%sy, 0.0%ni, 96.0%id, 1.0%wa, 0.0%hi,
> 0.0%si, 0.0%st
> Mem: 515192k total, 210700k used, 304492k free, 36488k buffers
> Swap: 489848k total, 18288k used, 471560k free, 117204k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 27647 fds 15 0 473m 59m 28m S 3.0 11.9 2:52.77 ns-slapd
>
>
> top - 11:48:26 up 25 days, 23:26, 2 users, load average: 0.02, 0.08,
> 0.10
> Tasks: 61 total, 1 running, 60 sleeping, 0 stopped, 0 zombie
> Cpu(s): 3.0%us, 0.0%sy, 0.0%ni, 97.0%id, 0.0%wa, 0.0%hi,
> 0.0%si, 0.0%st
> Mem: 515192k total, 222756k used, 292436k free, 36520k buffers
> Swap: 489848k total, 18288k used, 471560k free, 118932k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 27647 fds 15 0 483m 72m 30m S 0.0 14.4 4:12.04 ns-slapd
>
>
> top - 13:31:42 up 26 days, 1:09, 2 users, load average: 0.28, 0.17,
> 0.15
> Tasks: 61 total, 2 running, 59 sleeping, 0 stopped, 0 zombie
> Cpu(s): 1.1%us, 0.0%sy, 0.0%ni, 98.9%id, 0.0%wa, 0.0%hi,
> 0.0%si, 0.0%st
> Mem: 515192k total, 285572k used, 229620k free, 36540k buffers
> Swap: 489848k total, 18288k used, 471560k free, 140412k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 27647 fds 15 0 523m 116m 34m S 0.0 23.3 9:35.65 ns-slapd
Can you post your dse.ldif to pastebin.com? Be sure to omit or obscure
any sensitive data first. I'd like to see what all of your cache
settings are. Normally the server will increase in memory usage until
the caches are full, then memory usage should level off. The speed at
which this occurs depends on usage.
When the kernel kills your server, how much memory is it using? Is
there anything in the server error log at around the time the kernel
kills it?
Finally, if you are convinced that there is a real memory leak in the
server, would it be possible for you to run it under valgrind? Just
running it under valgrind for 30 minutes or so should reveal any memory
leaks in normal usage.
>
>>>>>
>>>>> can you give any help?
>>>>>
>>>>> thanks,
>>>>>
>>>>> KeeF
>>>>>
>>>>> Ryan Braun wrote:
>>>>>>>> A couple little bugs creeped up during the build. I think it
>>>>>>>> was during
>>>>>>>> the make install of ldapserver. One of the binaries (the first
>>>>>>>> one I
>>>>>>>> guess) was copied to /opt/dirsrv/bin (the bin being a file not a
>>>>>>>> directory) so the /opt/dirsrv/bin directory isn't getting
>>>>>>>> created. Quick
>>>>>>>> fix was just renaming /opt/dirsrv/bin to
>>>>>>>> /opt/dirsrv/bin.something and
>>>>>>>> rerunning make. Executing /opt/dirsrv/bin.something looks like
>>>>>>>> the binary
>>>>>>>> might be ldappasswd?
>>>>>>> Probably a bug in ds/mozldap/Makefile in the install section.
>>>>>>
>>>>>> I had a peek in there, it looks ok, but I'll add a mkdir -p
>>>>>> /opt/dirsrv/bin before the copy loop and see if that works next
>>>>>> time I build.
>>>>>>>> Second, there seems to be a missing library.
>>>>>>>>
>>>>>>>> Starting admin server . . .
>>>>>>>> output: ERROR: ld.so: object '/opt/dirsrv/lib/libssl3.so' from
>>>>>>>> LD_PRELOAD
>>>>>>>> cannot be preloaded: ignored.
>>>>>>>> output: apache2: Syntax error on line 123
>>>>>>>> of /opt/dirsrv/etc/dirsrv/admin-serv/httpd.conf: module
>>>>>>>> log_config_module
>>>>>>>> is built-in and can't be loaded
>>>>>>>> Could not start the admin server. Error: 256
>>>>>>>> Failed to create and configure the admin server
>>>>>>>> Exiting . . .
>>>>>>>>
>>>>>>>> I assumed the libssl3.so was supposed to be provided by
>>>>>>>> building nss from
>>>>>>>> source. So I just symlinked the system's libssl3.so provided by
>>>>>>>> libnss3-0d back to /opt/dirsrv/lib/.
>>>>>>> Ok. Or just edit the start-ds-admin script. Looks like a bug - it
>>>>>>> should use the correct path to libssl3.so. But then the NSS devel
>>>>>>> support in etch is not quite there.
>>>>>>
>>>>>> Gotcha
>>>>>>
>>>>>>>> Which leads me to my next question. The java components, are
>>>>>>>> they only
>>>>>>>> required for running the console on your client machines? So
>>>>>>>> building
>>>>>>>> with NOJAVA=1 will provide a fully working adminserver and
>>>>>>>> ldapserver, just no console binaries?
>>>>>>> Mostly correct. The only thing is that the way the console
>>>>>>> works, it
>>>>>>> downloads the ds and ds-admin jar files from the admin server.
>>>>>>> However,
>>>>>>> if you build them on the client machine and install them into
>>>>>>> $HOME/.fedora-idm-console/jars then the console will just use
>>>>>>> the local
>>>>>>> ones.
>>>>>>
>>>>>> Ok, well I tried installing the windows console on one of the
>>>>>> windows boxes around here (easier then downloading fc isos :) ),
>>>>>> fired up the console and am able to connect and it looks like it
>>>>>> wants to work, then it reports back that it can't find the
>>>>>> jars. So that being said, is there an easy way to use FC jars,
>>>>>> or do I need to build them for debian? (I have started trying to
>>>>>> build jss but am having some issues)
>>>>>>
>>>>>>>> To be honest, I haven't really looked into the different post
>>>>>>>> install
>>>>>>>> process' with 1.1.0 since 1.0.4 so the reason I could have missing
>>>>>>>> entries in the console could very well be my own fault :)
>>>>>>>>
>>>>>>>> Also, if I want to fine tune the location of some of
>>>>>>>> directories during
>>>>>>>> build. is it safe to modify the CONFIGURE_ARGS variable in the
>>>>>>>> adminserver and ldapserver's Makefile? I want to put
>>>>>>>> /opt/dirsrv/etc/dirsrv into /etc/dirsrv aswell as
>>>>>>>> /opt/dirsrv/var into
>>>>>>>> /var?
>>>>>>> Yes, for those components whose configure respect --sysconfdir and
>>>>>>> --localstatedir - which means not the mozilla components
>>>>>>> (mozldap, etc.)
>>>>>>> but everything else should work just fine. You'll also have to
>>>>>>> tweak
>>>>>>> the --prefix argument which is set by default.
>>>>>>
>>>>>> I'll play around with some options. I've started a wiki page for
>>>>>> the debian build. I don't have it linked onto the main page,
>>>>>> but you can check it out in recent changes.
>>>>>>
>>>>>> Ryan
>>>>>>
>
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3245 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/fedora-directory-users/attachments/20080305/bb245080/attachment.bin>
More information about the Fedora-directory-users
mailing list