[Freeipa-users] krb5kdc process at 100%

[Smith, Martin R. [smma0901@stcloudstate.edu]]

I removed the -w 4 from the config file. Here is what happens now. 

When a user with expired password logs in the krb5kdc process now crashes, instead of running at 100%. 
If I attach gdb to the process before it crashes and attempt to login the process doesn't crash. Here are the results of "bt"
---------
#0  0x00007fe84e0ea1d3 in __select_nocancel ()
    at ../sysdeps/unix/syscall-template.S:82
#1  0x00007fe84f2a8047 in krb5int_cm_call_select (in=<optimized out>,
    out=0x7fe8501d8780, sret=0x7fff421862b4) at sendto_kdc.c:564
#2  0x00007fe84ffd05ee in listen_and_process (handle=0x0,
    prog=0x7fff42187f52 "krb5kdc", reset=0x7fe84ffc6e10 <reset_for_hangup>)
    at net-server.c:1835
#3  0x00007fe84ffbcf68 in main (argc=3, argv=<optimized out>) at main.c:1069
--------

I have also attached the /var/log/krb5kdc

-Martin

-----Original Message-----
From: Simo Sorce [mailto:simo at redhat.com] 
Sent: Friday, September 09, 2011 8:56 AM
To: Smith, Martin R. [smma0901 at stcloudstate.edu]
Cc: freeipa-users at redhat.com
Subject: Re: [Freeipa-users] krb5kdc process at 100%

On Fri, 2011-09-09 at 05:09 +0000, Smith, Martin R.
[smma0901 at stcloudstate.edu] wrote:
> When I attach gdb to the process, I have tried the main process and 
> the four child processes, it provides no output.
> Here are the steps I'm taking:
>      1. On freeipa-server run htop and find the pid (or ps aux) 
>              1. Shows one parent PID and four child processes 
>                      1. 934 root 20   0 46784  2656   388 S  0.0  0.1
>                          0:00.00  `- /usr/sbin/krb5kdc
>                         -P /var/run/krb5kdc.pid -w 4
>                      2.  1939 root 20   0 78664  4460  2056 S  0.0
>                          0.1  0:00.26  |   `- /usr/sbin/krb5kdc
>                         -P /var/run/krb5kdc.pid -w 4
>                      3.  1938 root 20   0 78664  4460  2056 S  0.0
>                          0.1  0:00.26  |   `- /usr/sbin/krb5kdc
>                         -P /var/run/krb5kdc.pid -w 4
>                      4.  1936 root 20   0 78664  4460  2056 S  0.0
>                          0.1  0:00.26  |   `- /usr/sbin/krb5kdc
>                         -P /var/run/krb5kdc.pid -w 4
>                      5.  1935 root 20   0 78664  4212  1808 S  0.0
>                          0.1  0:00.26  |   `- /usr/sbin/krb5kdc
>                         -P /var/run/krb5kdc.pid -w 4
>              2. run sudo gdb 
>                      1. attach 934
>                      2. press "c"
>                      3. Wait for output… 
>      2. Attempt to login with user that has an expired password.
>      3. Now the krb5kdc process 934 starts running at 100% and the
>         user is unable to login. 
>      4. Only way to get the process back to normal is to type "service
>         ipa restart"

> 
> I've never debugged a program before so if I'm missing a step please 
> let me know.

Ok, let's simplify the problem first.

apperently you have a quadcore cpu so by default we configured krb5kdc to spawn 4 worker processes. Let's bring it down to not spawning any worker process so we can simplify debugging.

Go to /etc/sysconfig/krb5kdc and remove the "-w 4" argument from it.

Then simply do a service krb5kdc restart (no need to restart the whole ipa service for this).


If krb5kdc locks up again, gdb the process like you have done before but do not press c, type 'bt' instead and copy the log then you can exit gdb.

Simo.


-- 

Simo Sorce * Red Hat, Inc * New York

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: krb5kdc.txt
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20110909/d41573ed/attachment.txt>