[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Linux-cluster] GFS2 lock accumulation

Hi All,

I've been running GFS and now GFS2 for several years on a two-node mail cluster, generally with good results, especially once GFS2 became production ready and we upgraded. However from time to time (ranging from a few days to a month), we'll get a "stuck" lock on one particular file or another which them blocks a user from their mail. While looking into this, I've recently become aware of a VERY large number of glocks being left behind after our nightly rsync backups. I'm checking on the lock situation with "gfs2_tool lockdump /home" and counting locks by piping through "grep ^G | wc -l". We have two GFS2 filesystems mounted. On one of them, the number of glocks returns to "normal" after the backup (currently showing about 5400.) On the other, it stays very high although it will drop somewhat throughout the day. Currently I am seeing over 500,000. Given the ten minutes or so that it takes to list them, this seems like it can't be great for performance.

Most of the locks look like this:

G:  s:SH n:5/b25806 f: t:SH d:EX/0 l:0 a:0 r:3
H: s:SH f:EH e:0 p:31042 [(ended)] gfs2_inode_lookup+0x114/0x1f0 [gfs2]

Note that the pid (31042 in this case) corresponds to one of the completed rsync processes which generated the locks in the first place.

My questions are 1) Is this a bad thing? My gut feeling is "yes" but perhaps the system is highly efficient in dealing with these locks, and 2) Can anything be done about it? The tuning opportunities in GFS2 are very limited compared to GFS, and the few things I've tried seem to have no effect.

By the way, I am running with plock_ownership="1" and plock_rate_limit="0" in cluster.conf.

Thanks in advance,

Allen Belletti
allen isye gatech edu                             404-894-6221 Phone
Industrial and Systems Engineering                404-385-2988 Fax
Georgia Institute of Technology

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]