Joshua Baker-LePain wrote:
On Tue, 25 Oct 2005 at 10:12am, Kostas Georgiou wroteWith 3ware cards i get slightly better results (for our workload) in RHEL4with the following settings: vm.dirty_expire_centisecs = 1000 vm.dirty_writeback_centisecs = 500 vm.dirty_ratio = 5 vm.dirty_background_ratio = 10These numbers had no effect for me, but I'm likely to play with 'em a bit more.
I've got an 3ware 7500-8 card yesterday that I had to configure for RAID-5, and experienced exactly the same lack of performance Joshua is having problems with. I know that 7500-8 isn't RAID-5 speed champion (not even close), but I believe the card itself should be faster than what I got (around 18 MB/sec for ext3 and 35 MB/sec for ext2).
The above settings also make almost no difference to me. What made difference was switching from ext3 to ext2, almost double write speed, instantly. The read speed stayed the same. Interestingly, the rewrite Bonnie test gave almost identical speed with both ext2 and ext3.
I also did a speed test using dd, simply writing some data to raw partition using relatively large block size. For 5-disk RAID-5 volume I got 37 MB/sec, which is quite close to what Bonnie measured for ext2.
In my setup, I have two volumes on 3ware, one RAID-1 for OS (two drives), and one RAID-5 for data (using five drives). The thing I noticed was that when running Bonnie tests on RAID-5 volume, the RAID-1 volume suffers badly. For example, while Bonnie runs "intelligent" read/write tests, it is almost impossible to login into the box using ssh. With getc/putc tests it is a bit better, but still slow.
Another thing I noticed is that even the RAID-1 volume has a bit slower performance than expected when reading. While write speed was OK for a simple mirror (around 30 megs per second), the read speed was also about the same (I would expect it to be 50+, looking at the write speed).
Have you tried the kernel in the following bugzilla? I didn't had the time to test it but it seems promising. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=156437 (Writing large file to 1TB ext3 volume sometimes very slow)I *did* try that kernel, to no improvement.
Mentioned kernel (2.6.9-22 with a patch) did made some improvements in my case. While the read/write speed was still about the same, the machine become responsive while Bonnie was hammering the RAID-5 volume with getc/putc tests (almost as if Bonnie was not running). During the "intelligent" read/write tests, it was still a bit unresponsive, but it was possible to login onto the box using ssh (after a long delay to get the password prompt), so it wasn't as bad as without a patch.
When creating filesystem, I used -R stride=16 -T largefile4 options (its going to be only relatively large files stored there). Looking at the man page, 16 should be correct value for 64kB stripe RAID-5 volume (64kB divided by 4kB block size). Or I got it wrong?