[Cluster-devel] Re: [PATCH 2/2] dlm: Add down/up_write_non_owner to keep lockdep happy

[Steven Whitehouse]

Hi,

On Thu, 2009-11-12 at 12:34 -0600, David Teigland wrote:
> On Thu, Nov 12, 2009 at 05:24:12PM +0000, Steven Whitehouse wrote:
> > > > Nov 12 15:10:01 chywoon kernel: [ INFO: possible recursive locking
> > > > detected ]
> > > 
> > > That recursive locking trace is something different.  up_write_non_owner()
> > > addresses this trace, which as you say, is from doing the down and up from
> > > different threads (which is the intention):
> > > 
> > I don't think it is different, the traces differ due to the ordering of
> > running of dlm_recoverd and mount.gfs2,
> 
> I explained the "recursive locking" warning back in Sep:
> 
>  > I've not looked at how to remove this "recursive" message.  What
>  > happens is that mount calls dlm_new_lockspace() which returns with
>  > in_recovery locked.  mount then makes a lock request which blocks on
>  > in_recovery (as expected) until the dlm_recoverd thread completes
>  > recovery and releases the in_recovery lock (triggering the unlock
>  > balance) to allow locking activity.
> 
> It doesn't appear to me that up_write_non_owner() would suppress that.
> 
> Dave
> 
It is simply down to the ordering of the running of the threads as to
which message you get at mount time. There are two possible scenarios:

Scenario 1:

1. mount.gfs2 calls (via mount sys call and gfs2) dlm_newlockspace()
which takes the ls_in_recovery rwsem with a down_write()
2. mount.gfs2 goes on to try and take out a lock on the filesystem, and
calls dlm_lock which tries to do a down_read() on the rwsem. Since this
is from the same thread as the down_write() you get the recursive
locking message reported in the dmesg which I attached to my earlier
email.

In the second scenario, dlm_recoverd runs between step 1 and 2 above.
this results in the trace which you reported, since ls_in_recovery has
then been unlocked from a different thread, which creates the unlocking
balance trace which you posted.

In both cases the cause is the same, its just the running order of the
threads which results in it being reported in a different way. The patch
should fix both of these reports, since it annotates the up & down write
side of the rwsem,

Steve.