100Mbps Ethernet Speed/Efficiency

[Joel Jaeggli]

On Tue, 6 Apr 2004, Neall wrote:

> On Tue, 6 Apr 2004 at 3:38pm, Joel Jaeggli <joelja at darkwing.uoregon.edu>...:
> 
> > You can build a pretty wicked disk subsystem these days. single 15k rpm
> > disk-drives can read a rate of around 86-78MB/s depending on what part of
> 
> Technology Marches on!  My keeping up with it doesn't :)
> 
> > Do the machines in question have the ethernet in 64 bit pci slots?
> 
> It's an SBC, and ethernet is on board.  The schematic shows an
> interconnect via 64-bit PCI-X, 133 MHz, via an Intel 82870P2 PCI/PCI-X
> hub.  The eth chip itself is an Intel 82546EB.  Dual Gigabit.  Soldered
> in.  Copper trace PCI, but no plug-in connectors.
> 
> > block reads on 4GB files from a netap-940 we were testing were order of
> > 83MB/s using jumbo 9k ethernet frames... dropping that to 1500mtu made it
> 
> Off topic (SORRY):  Where do I tune these parameters in my NFS setup?
> I'm actually running RHEL WS3.0 (not Fedora) but should be similar.

mtu is actaully tuned on a per-interface basis. you want be be careful and 
avoid situations where you might blackhole yourself talking to 1500mtu 
devices on the same subnet, so generally when we want to use a 9k mtu we 
do so on a private switch or in the case of our larger switches on a 
private vlan. 

just doing: 

ifconfig ethX mtu 9000 

on the interfaces you want to talk over is enough to get the ball rolling 
assuming your switch support jumbo frames (not all do). a good cheap 
switch which does is the smc SMC8508T which is < 1$60.

nfs is one of the few applications in our testing where we see a fairly 
serious performance improvement in going to jumbo frames.

> > desktop which just has 100baset. For small switched environments where
> > performance is important unmanged gig copper switches now cost order of
> > $20 a port or less, so if you need the speed it may be cheap.
> 
> Exactly, our plan is it run multiple SBCs but not have them communicate
> over a backplane.  Instead, using an unmanaged, copper giga switch and
> Cat5E between them. :)  Endpoints will be NFS-mounted RAM disks.  You
> folks are demonstrating to me that it's a pretty efficient pipeline.

should be.

> Neall
> 

-- 
-------------------------------------------------------------------------- 
Joel Jaeggli  	       Unix Consulting 	       joelja at darkwing.uoregon.edu    
GPG Key Fingerprint:     5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2