Re: Fileserver optimizations

[Brian Long <brilong cisco com>]

On Wed, 2005-10-19 at 10:49 -0400, Joshua Baker-LePain wrote:
> I've got a couple of "old" fileservers based on 3ware 7500-8 boards that 
> I'm updating to centos-4.  The servers had been running RH7.3 (!!) with 
> SGI's XFS supporting kernel, but I've about given up hope on RH seeing the 
> light on XFS.  So, I dumped everything to tape, wiped, reinstalled, 
> formatted with ext3, and then the troubles began.  Performance is 
> *abysmal* compared to the original configuration.  I'm hoping I just need 
> to do some tuning, so I'm looking for some tips.
> 
> Here's the setup.  Each system has 2 3ware 7500-8 boards in it (on 
> independent PCI buses), and the boards are configured for hardware RAID5 
> with a hot spare.  Then I do a RAID0 stripe across the 2 arrays.  In the 
> old setup, I was getting >100MB/s writes and >300MB/s reads.  With 
> centos-4 and ext3, I'm getting ~30MB/s writes and ~200MB/s reads.  Yick.
> 
> For comparison, I also tested with a 15 disk software RAID6.  That gets 
> ~100MB/s writes but only ~150MB/s reads.
> 
> Has anyone got any tuning suggestions for fileserver performance?  In the 
> hardware RAID case (which I'd like to stick to if possible, although it's 
> looking doubtful), I've played with 'blockdev --setra' on the 3ware 
> devices without much effect.  Anything else I can look to tuning?  These 
> servers tend to handly mostly larger files.

Joshua,

I imagine the tools you're using on CentOS 4 are just more accurate in
measuring IO  :)  30MB/sec RAID-5 write and 200MB/sec read is not too
bad depending on the number of spindles.  What tools are you using to
measure performance?  If the tools are part of the old and new OS, I
imagine those tools have gone through significant changes so you cannot
compare the results directly.  Have you measured actual application or
other performance (i.e. NFS or whatever)?

/Brian/

-- 
       Brian Long                      |         |           |
       IT Data Center Systems          |       .|||.       .|||.
       Cisco Linux Developer           |   ..:|||||||:...:|||||||:..
       Phone: (919) 392-7363           |   C i s c o   S y s t e m s