[Linux-cluster] disk performance issues

[Terry]

Hello,

I am starting here because the filesystems in question are GFS.  Here
are a few facts:

iscsi
bonded ethernet with 9000 MTU setting
LVM
GFS

Here is an iostat of the volumes in question:
Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz
avgqu-sz   await  svctm  %util
sdc              11.36   564.51 373.65 99.31  3082.85  5397.92
17.93     7.64   16.13   1.69  79.82
sdd              11.92   567.96 266.21 48.28  2230.24  5044.40
23.13     0.91   51.38   2.04  64.30
sdf              13.80   536.48 263.91 77.40  2225.89  5014.98
21.21     0.70   46.72   2.13  72.54
sde              39.97   802.23 345.06 47.22  3089.78  6969.91
25.64     2.42   45.02   1.89  74.32

The load average of the box is 40.   I am not using near the capacity
of the network.

Based on all of this, anyone have any thoughts?  My await seems a tad high.  :)

Thanks!