Re: slow NFS performance, especially during writes

["Eric Gunnett" <eric zoovy com>]

	A silly thing but I would check the duplex of the ports on the clients verify that the server and the port are both set to the same duplex and speed. We had a similar problem with the writes taking forever because of the duplex. You also may also want to see the output from the nfsstat -c for the client you are on and nfsstat -s for the server they can lend a hand in spotting the bottlenecks.



Eric Gunnett
System Administrator
Zoovy, Inc.
eric zoovy com


>>> tom georgoulias motorola com 05/15/03 02:32PM >>>
I'm trying to diagnose a problem with slow NFS performance (and unhappy 
users ;), especially during write operations to the NFS mounted 
filesytems.

I'm running a PE 2650 (2 x 2.4GHz with 6GB RAM, Power Vault 220S 
attached via PERC3/DC) and Red Hat 7.3 (with all errata applied).

I've got about 200 clients accessing data on this system, 95% of which 
are solaris 8 systems, and I'd characterize it as a heavy load for read 
and write. The performance of the server seems especially bad during 
write operations (i.e. saving edits to text files, untarring large tar 
balls).  We also see a bunch of "NFS server not responding still trying" 
"NFS server ok" messages in the client's consoles.

The network bandwidth to the server doesn't seem to be a problem:

bender:apollo$ ping -s eclipse0
PING eclipse0: 56 data bytes
64 bytes from eclipse0 (163.10.191.76): icmp_seq=0. time=0. ms
64 bytes from eclipse0 (163.10.191.76): icmp_seq=1. time=0. ms
64 bytes from eclipse0 (163.10.191.76): icmp_seq=2. time=0. ms
64 bytes from eclipse0 (163.10.191.76): icmp_seq=3. time=0. ms
^C
----eclipse0 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms)  min/avg/max = 0/0/0
bender:apollo$

I am using a mostly default NFS settings for a RHL7.3 install, with the 
following exceptions:   On the advice of the NFS FAQ, I have changed 
/etc/exports to include "sync" for each exported filesystem to maintain 
.  Other performance changes include increasing the rsize/wsize to 8192 
and running 64 nfsd instances. Based on what I read in NFS-HOWTO, I 
checked /usr/src/linux-2.4/include/linux/nfsd/const.h and found that 
NFSSVC_MAXBLKSIZE=8192, so it seems I'll have to rebuild a kernel to get 
that as well.  No good at this point.

Other changes I considered making are increased to the input queue 
(describe here 
http://nfs.sourceforge.net/nfs-howto/performance.html#MEMLIMITS), but 
I'm scared of the line that says "...machines have been reported to 
crash if these values are left changed for long periods of time."

This is the output from nfsstat -m from a typical solaris client and 
shows the options in place for the server and for a solaris NFS server.

Linux:
/soc/users/copperhead from eclipse0:/export/users/copperhead
  Flags: 
vers=3,proto=udp,sec=none,hard,intr,link,symlink,rsize=8192,wsize=8192,retrans=5,timeo=11
  Attr cache:    acregmin=3,acregmax=60,acdirmin=30,acdirmax=60
  Lookups:     srtt=7 (17ms), dev=3 (15ms), cur=2 (40ms)
  Reads:     srtt=9 (22ms), dev=7 (35ms), cur=4 (80ms)
  Writes:     srtt=18 (45ms), dev=12 (60ms), cur=8 (160ms)
  All:     srtt=7 (17ms), dev=3 (15ms), cur=2 (40ms)

Solaris:

/usr/local_design from macbeth:/fsys/fs16/local_design
  Flags: 
vers=3,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
  Attr cache:    acregmin=3,acregmax=60,acdirmin=30,acdirmax=60

The most notable differences seem to be the protocol, r/w sizes, and 
timeo.  I want to boost the performance of this system in the best, 
safest way, but I'm running out of "know how" on what I should be 
looking at.  Should I increase the input queue sizes?  Does RHL 7.3 do 
NFS over TCP, and could I do that and revert to using the "async" in 
/etc/exports?  What are the best options for me in tuning this system? 
Any help provided is *greatly* appreciated.

Tom


_______________________________________________
Valhalla-list mailing list
Valhalla-list redhat com 
https://www.redhat.com/mailman/listinfo/valhalla-list