[Linux-cluster] Timeout causing GFS filesystem inaccessibility

[Rich Paredes <rparedes gmail com>]

Assumptions: 3 node cluster. 
All 3 nodes are lock managers
Nodes 1 and 2 mount GFS filesystems
Node 1 during failure is master, node 2 and node 3 are slaves

Error on node 2 is:
lock_gulmd_LT000[3608]: Timeout (15000000) on idx: 2 fd:7 (node1:192.168.101.11)

This error keeps repeating in the logs and GFS filesystem are totally
inaccessible.  To fix, the master lock manager needs to be manually
expired and then rebooted because applications were accessing GFS
filesystems.

It looks like error message is generated from lock_io.c.

Does anyone know exactly what causes this error?

Thanks,

Rich