[Linux-cluster] optimising DLM speed?

[Alan Brown]

Steve:

To add some interest (and give you numbers to work with as far as dlm 
config tuning goes), here are a selection of real world lock figures 
from our file cluster (cat $d | wc -l)

/sys/kernel/debug/dlm/WwwHome-gfs2_locks  162299  (webserver exports)
/sys/kernel/debug/dlm/soft2-gfs2_locks  198890  (Mainly IDL software - 
it's hopelessly inefficient, 32Gb partition)
/sys/kernel/debug/dlm/home-gfs2_locks  74649 (users' /home directories, 
150Gb partition)
/sys/kernel/debug/dlm/User1_locks  318337 (thunderbird, mozilla, 
openoffice caches, 200gb partition)
/sys/kernel/debug/dlm/Peace04-gfs2_locks  265955  (solar wind data)
/sys/kernel/debug/dlm/Peace05-gfs2_locks  332267
/sys/kernel/debug/dlm/Peace06-gfs2_locks  283588

At the other end of the spectrum:

/sys/kernel/debug/dlm/xray0-gfs2_locks  24917 (solar observation data)
/sys/kernel/debug/dlm/xray2-gfs2_locks  558
/sys/kernel/debug/dlm/cassini2-gfs2_locks  598 (cassini probe data from 
Saturn)
/sys/kernel/debug/dlm/cassini3-gfs2_locks  80
/sys/kernel/debug/dlm/cassini4-gfs2_locks  246
/sys/kernel/debug/dlm/rgoplates-gfs2_locks 27 (global archive of 100 
years' worth of photographic plates from Greenwich observatory)


Directories may have up to 90k entries in them, although we try very 
hard to encourage users to use nested structures and keep directories 
below 1000 entries for human readability (exceptions tend to be mirrors 
of offsite archives), but the counterpoint to is that it drives the 
number of directories up - which is why I was asking about the 
dirtbl_size entry.

~98% of directories are below 4000 entries.

FSes usually have 400k-2M inodes in use.

Does that help with tuning recommendations?