[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] CLVM/GFS2 distributed locking



Here is a simple C utility for locking file - it's combination of two sources:
- reading lock info from here: http://uw714doc.sco.com/en/SDK_sysprog/_Getting_Lock_Information.html
- acquiring file lock from here: http://siber.cankaya.edu.tr/ozdogan/SystemsProgramming/ceng425/node161.html

Now, to make use of it.

Regards,
Stevo.

On Wed, Jan 4, 2012 at 6:10 PM, Stevo Slavić <sslavic gmail com> wrote:
Hello Steven,

I guess license covers only 5.4. Anyway I'm just told it's not an option at the moment to do the upgrade.


About locking used, ActiveMQ uses Java 6 standard API for trying to acquire file lock, here is javadoc for the method used:

http://docs.oracle.com/javase/6/docs/api/java/nio/channels/FileChannel.html#tryLock%28long,%20long,%20boolean%29

ActiveMQ tries to obtain non-shared (thus exclusive) lock on whole file on shared file system, with range from 0 to 1, since the file being locked is empty. As documentation states, tryLock is non-blocking, it executes immediately. If ActiveMQ fails to obtain a lock it will  loop (pause and retry acquiring lock again) until lock is obtained. In initial state first node obtains lock and becomes master, second one fails to obtain lock and gets into this loop, as expected. Problem is that slave ActiveMQ on cannot obtain a lock even after first node gets fenced - it reports that the file on shared storage is still locked. Simple custom java tool that I made reports the same, that the file is locked.

OpenJDK 1.6 update 20 is being used as Java runtime. I haven't yet found in openjdk source exact code which tryLock will call on Linux.


Is there non-Java tool that could be used to reliably check if a file (on gfs2) is (or can be) exclusively locked (regardless of where the process holding lock is  running, on same or different node where the tool is being run)?


Regards,
Stevo.




On Tue, Jan 3, 2012 at 10:59 AM, Steven Whitehouse <swhiteho redhat com> wrote:
Hi,

On Fri, 2011-12-30 at 14:39 +0100, Stevo Slavić wrote:
> Hello RedHat Linux cluster community,
>
> I'm in process of configuring shared filesystem storage master/slave
> Apache ActiveMQ setup. For it to work, it requires reliable
> distributed locking - master is node that holds exclusive lock on a
> file on shared filesystem storage.
>
How does it do this locking? There are several possible ways this might
be done, and some will work better than others.

> On RHEL (5.4), using CLVM with GFS2 is one of the options that should
> work.
Why are you using RHEL 5.4 and not something more recent? Note that if
you are a Red Hat customer, then you should contact our support team who
will be very happy to assist.

> Third party configured the CLVM/GFS2. I'd like to make sure that
> distributed locking works OK.
> What are my options for verifying this?
>
I think we need to verify which type of locking the application uses
before we can answer this,

Steve.

> Thanks in advance!
>
> Regards,
> Stevo.


#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int main (int args, char* argv[])
{
  char* file = argv[1];
  int fd;
  struct flock lock;
  struct flock lck;
  
  printf("opening %s\n", file);
  /* Open a file descriptor to the file. */
  fd = open(file, O_WRONLY);
  printf("checking for locks\n");
  (void) printf("   pid type    start   length\n");
  lck.l_start = 0L;
  do {
    lck.l_whence = 0;
    lck.l_len = 0L;
    lck.l_type = F_WRLCK;
    (void) fcntl(fd, F_GETLK, &lck);
    if (lck.l_type != F_UNLCK) {
      (void) printf(" %5d   %c  %8d %8d\n",
		    lck.l_pid,
		    (lck.l_type == F_WRLCK) ? 'W' : 'R',
		    lck.l_start,
		    lck.l_len);
		    /* if this lock goes to the end of the address
		     * space, no need to look further, so break out.
		     */
		    if (lck.l_len == 0)
		      break;
		    /* otherwise, look for new lock after the one
		     * just found.
		     */
		    lck.l_start += lck.l_len;
      
    }
  } while (lck.l_type != F_UNLCK);
  printf("locking\n");
  /* Initialize the flock structure. */
  memset(&lock, 0, sizeof(lock));
  lock.l_type = F_WRLCK;
  /* Place a write lock on the file. */
  fcntl(fd, F_SETLKW, &lock);
  printf("locked; hit enter to unlock...");
  /* Wait for the user to hit enter. */
  getchar();
  printf("unlocking\n");
  /* Release the lock. */
  lock.l_type = F_UNLCK;
  fcntl(fd, F_SETLKW, &lock);
  close(fd);
  return 0;
}

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]