Problem in PTHREAD_PROCESS_SHARED rwlocks?

["Philip Beevers" <philip beevers ntlworld com>]

(N.B. This is my first post to the list - apologies if I've come to the
wrong place)

I've been doing some work with the NPTL shipped with RH9. It's very good
- I've found it reliable, scalable and compatible in my testing so far.

However, I think I've found a problem with inter-process rwlocks. I've
listed my test case below, which is similar in form to tst-rwlock4.c,
but it does a bit more locking and unlocking in the parent and child
processes - basically, the parent and child should contend for a write
lock, then share a read lock, then bounce around these locks ad
infinitum.

Unfortunately, when the second process tries to gain the write lock
(whilst it's already locked by the first) I get an error - here's the
output of my test:


7360: Write locking
7360: Locked
7359: Write locking
7359: Lock failed (35/Resource deadlock avoided)


Looking in the source, it seems like this error is returned if it thinks
the locking thread already owns the write lock - but in this case, it
doesn't. Any ideas? Here's the source (apologies, it's a bit long, but I
thought I'd better check all the returns in case I messed up somewhere):


#include <errno.h>
#include <fcntl.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>

#define FL_SZ 8192

void SysError(const char* fn, int err)
{
    printf("Error in %s - %d (%s)\n", fn, err, strerror(err));
    exit(1);
}

void CheckAndUnlock(int err, pthread_rwlock_t* rwlock)
{
    if (err != 0)
    {
        printf("%d: Lock failed (%d/%s)\n", getpid(), err,
strerror(err));
        sleep(5);
    }
    else
    {
        printf("%d: Locked\n", getpid());

        sleep(10);

        printf("%d: Unlocking\n", getpid());
        err = pthread_rwlock_unlock(rwlock);

        if (err != 0)
        {
            printf("%d: Unlock failed (%d/%s)\n", getpid(), err,
strerror(err));
        }
    }
}

int main(int argc, char** argv)
{
    // Create shared read-writer lock in parent process
    pthread_rwlock_t* rwlock = NULL;
    int fd = open("/tmp/myfile", O_CREAT | O_TRUNC | O_RDWR, 0666);

    if (fd < 0)
    {
        SysError("open", errno);
    }

    if (ftruncate(fd, FL_SZ) < 0)
    {
        SysError("ftruncate", errno);
    }

    void* ptr = mmap(NULL,
                     FL_SZ,
                     PROT_READ | PROT_WRITE,
                     MAP_SHARED,
                     fd,
                     0);

    if (ptr == MAP_FAILED)
    {
        SysError("mmap", errno);
    }

    rwlock = (pthread_rwlock_t*)ptr;
    pthread_rwlockattr_t attr;
    int err = pthread_rwlockattr_init(&attr);

    if (err != 0)
    {
        SysError("pthread_rwlockattr_init", err);
    }

    err = pthread_rwlockattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);

    if (err != 0)
    {
        SysError("pthread_rwlockattr_setpshared", err);
    }

    err = pthread_rwlock_init(rwlock, &attr);

    if (err != 0)
    {
        SysError("pthread_rwlock_init", err);
    }

    // Now fork so both parent and child are locking and unlocking
    if (fork() < 0)
    {
        SysError("fork", errno);
    }

    while (1)
    {
        printf("%d: Write locking\n", getpid());
        CheckAndUnlock(pthread_rwlock_wrlock(rwlock), rwlock);

        sleep(1);

        printf("%d: Read locking\n", getpid());
        CheckAndUnlock(pthread_rwlock_rdlock(rwlock), rwlock);
    }
}


(This is an out-of-the-box RH9 install - the kernel version is
2.4.20-8smp, glibc version is 2.3.2-11.9, and it's running on a
Xeon-based machine)

Although it's no guarantee that my code's not to blame, it works fine on
Solaris, producing the kind of output I'm expecting:


3448: Write locking
3448: Locked
3447: Write locking
3448: Unlocking
3447: Locked
3448: Read locking
3447: Unlocking
3448: Locked
3447: Read locking
3447: Locked
3448: Unlocking
3448: Write locking
3447: Unlocking
3447: Write locking
3447: Locked


BTW, inter-process mutexes and cond vars work well - I've had no
problems with them.

Thanks in advance for any help - and again, apologies if I'm wasting
anyone's time by posting in the wrong place. Feel free to redirect me if
necessary!


-- 
Phil