[Linux-cluster] Re: D Apache processes

[Axel Thimm]

On Mon, Dec 06, 2004 at 07:25:32PM -0800, Duncan Morgan wrote:
> Unfortunately we don't see the load decrease at all and have to reboot
> all servers in the cluster - causing mass chaos :)) so I'm not sure we
> are talking about the same issue.

Then you have enough time to examine the problem more thoroughly. Use
lsof to check which resources are accessed by the processes in
D-state. You will probably find that a larger share is accessing the
same files.

> Thanks for your input.
> 
> Duncan
> 
> From: linux-cluster-bounces at redhat.com
> [mailto:linux-cluster-bounces at redhat.com] On Behalf Of Axel Thimm
> Sent: Monday, December 06, 2004 7:15 PM
> To: linux clistering
> Subject: [Linux-cluster] Re: D Apache processes
> 
> On Mon, Dec 06, 2004 at 06:52:02PM -0800, Duncan Morgan wrote:
> > Hello,
> > 
> > We have apache running on 14 GFS nodes where the web roots are shared
> > via GFS. Occasionally we see that the load on all nodes rises
> > dramatically (to 150+) and all httpd processes become dead (D). I know
> > this is a little lacking in details but does anyone have any insight
> > into this? We suspected perhaps a cron job was running simultaneously
> > against the GFS file system on all nodes but have virtually ruled this
> > out.
> 
> I am getting the same behaviour on a non-clustered, non GFS system
> (dual opteron on FC2/x86_64). There is a peak of almost all httpd
> processes in D-state (that's not really dead, but "uninterruptible
> sleep", e.g. when the kernel does IO for the userland process). A few
> seconds later the number of D-processes fall down to less than a dozen
> and you can watch your load exponentially decrease.
> 
> Note that 150 is the default MaxClient setting for apache, that's why
> you get slightly more than 150 load.
> 
> I guess that's a kernel issue with too many processes accessing the
> same files. Nothing to do directly with GFS.
> 
> > Please help - this is very alarming.
> > 
> > Thanks in advance,
> > Duncan Morgan
> > 
> 

-- 
Axel.Thimm at ATrpms.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20041207/cb6fed1e/attachment.sig>