RE: Recommended Partitioning Scheme for a server.

["Waldher, Travis R" <Travis R Waldher boeing com>]

> -----Original Message-----
> From: Rick Stevens [mailto:rstevens vitalstream com] 
> Sent: Monday, November 17, 2003 5:42 PM
> To: redhat-install-list redhat com
> Subject: Re: Recommended Partitioning Scheme for a server.

<snip> 

> >>Well, speaking as a sysadmin with about 250 Linux servers
> >>(everything from streaming and web servers to database 
> >>systems and massive LDAP servers), the second partitioning 
> >>scheme permits much better management over how a server is 
> >>used and its performance.
> > 
> > 
> > Well.. That's a better answer than I got, but that still leaves me 
> > with the question.  Why?
> 
> I gave the reasons as 1), 2), 3), and 4) in my reply, which 
> you quote below.

Ok.. I'll rephrase, maybe I'm not asking the right question.  

With the newer systems today, the faster disks, the faster processors,
more reliable disks.  Are the issues of rebuild time (thinking fsck,
even tape restore), access time, etc. on even a 9GB system partition
with everything on it, still an issue or is it more because UNIX systems
have been traditionally built that way?  (I'm not trying to argue, I'm
trying to understand why things are done the way they are done, and if
it is "a tradition" thing, does it still apply with today's faster
computer systems?)

Examples - if all my eggs were in one basket for the root filesystem and
the filesystem craps out, say even 5-6GB of data will only take about
20-25 minutes to restore (tape drive running about 5MB/s).

<snip>

> 
> > If this was a shared workstation, a workstation, or an application 
> > server I probably wouldn't be asking this, but I want the NFS file 
> > Server to be stable, reliable, but also as simple as possible.
> 
> Then make it all one partition and install the absolute 
> minimum stuff to make the NFS/NIS stuff work.  You'd only 
> need a fairly small partition, say 4GB, with that sort of layout.
> 
> >>You want separate partitions for a number of good reasons:
> >>
> >>1)	If a filesystem craps out, you only have to restore the data for
> >>that specific filesystem, not the entire damned machine.
> > 
> > 
> > Agreed, Just in this case, the filesystem that keeps the machine 
> > running is only about 3GB of real data.  The actual users data is 
> > going to be on 15-30 different filesystems when deployed.  
> To spread 
> > the risk of that happening.  For example:
> > 
> > Filesystem            Size  Used Avail Use% Mounted on
> > /dev/cciss/c0d0p2     6.7G  2.1G  4.3G  33% /
> > none                  503M     0  503M   0% /dev/shm
> > /dev/cciss/c0d2p1     734G  3.3G  693G   1% 
> > /dev/vg00/lvol01       52G   44G  6.2G  88% (mount points 
> been deleted
> > for security reasons)
> > /dev/vg00/lvol02       43G   40G  1.3G  97% 
> > /dev/vg00/lvol03      2.9G  1.9G  921M  68% 
> > /dev/vg00/lvol04       45G   34G   10G  77% 
> > /dev/vg00/lvol05      4.8G  3.4G  1.2G  73% 
> > /dev/vg00/lvol06      960M   76M  836M   9% 
> > /dev/vg00/lvol07      1.4G  1.1G  327M  77% 
> > /dev/vg00/lvol08      1.9G  1.2G  669M  65% 
> > /dev/vg00/lvol09      2.0G  1.6G  335M  83% 
> > /dev/vg00/lvol10      387M  301M   67M  82% 
> > /dev/vg00/lvol11       84G   75G  7.1G  92% 
> > /dev/vg00/lvol12       42G   35G  6.1G  85% 
> > /dev/vg00/lvol13      3.3G  2.2G  929M  71% 
> > /dev/vg00/lvol14      2.9G  2.4G  390M  87% 
> > /dev/vg00/lvol15      496M   33M  443M   7% /tmp
> > /dev/vg00/lvol16      6.8G  5.7G  796M  88% 
> > /dev/vg00/lvol17      4.6G  2.9G  1.5G  66% 
> > 
> > (oh.. Don't mind the 734GB drive, it's not staying like that, it's 
> > only that way to play around with it real quick.)
> 
> Perfect.  That's the way to do it.  I assume you're using 
> some form of hash to sort out who's directory goes where?

Hmm.. I'm guessing I did that, but haven't heard that term before.  What
do you mean by "some form of hash"?

> 
> >>2)	The bigger the filesystem, the harder the disk system has to
> >>work to use it.  This slows down disk access, both in read
> >>and write performance.
> >>
> > 
> > 
> > At which point do you normally start seeing a performance 
> loss for an 
> > NFS file server? 100MB? 5GB? 50GB?
> 
> It rather depends on the underlying filesystem.  Remember, 
> NFS is a transport layer, not a true filesystem.  Your 
> question should really be "Where do I start seeing 
> performance loss using (insert underlying filesystem here) 
> filesystems?"  With ext3 filesystems, I start seeing 
> performance problems with 7200 RPM IDE drives around the 75GB 
> mark. SCSI is a bit better, 7200 RPM problems show up around 
> the 90-100GB spot.  10K RPM drives fair better, but they're expensive.

I have either 36GB drives or 72GB drives that are 10,000RPM Ultra3 SCSI.
One hardware RAID for example (what vg00 resides on) is 9-36GB hard
disks RAID5 with a hot spare, on a controller with 256MB cache (128
Read/128 Write).  The "bigdrive" is a hardware RAID5 ADG of 13-72GB
10,000RPM Ultra3 SCSI disks with one hot spare. A future vg01 will
reside on that one.

The system (root disk) itself is a pair of hardware mirrored 9GB 10,000
RPM Ultra3 SCSI disks.

> 
> >>3)	The bigger a filesystem is, the larger the block sizes will be
> >>to match up to the number of inodes available.  This means
> >>that you will end up wasting a lot of disk because small 
> >>files will suck up a big 
> >>block.  For example, the normal block size would be 2KB.  If 
> >>you have a 4-byte file, it sucks up one block or 2KB of disk, 
> >>wasting 2044 bytes of disk.  If your block size is 16KB, then 
> >>that same 4-byte file now occupies 16KB of disk and wastes 
> >>16,380 bytes of space.
> >>
> > 
> > 
> > Agreed, that's one thing I generally keep in mind.
> 
> Good.  That's a hard concept for many folk to wrap their 
> brains around.

My only big problem is my mix of data.  It's a PITA trying to find a
happy medium on the block size and unfortunately it's all intermingled,
and not something I have the time or energy to straighten out.  (there's
10-20 years of history on that)

<snip>

> 
> If you use disk druid, set up your other partitions and "use 
> rest of disk" for /usr.

Disk druid? Whats that?  I use fdisk. :)

That Disk druid utility has issues with LVM sometimes that causes it to
crash during setup.  Besides, I feel that using fdisk allows you to
become closer to the server you are supporting, building that emotional
bond that is required to keep me and the server...    Ohh.. WAY to much
info there... Sorry.. hehe


> 
> >>	/proc - 2GB (currently 919MB)
> >>
> >>Bad idea.  DO NOT CREATE A /proc PARTITION!  /proc is a
> >>virtual filesystem.  It doesn't really exist.  It is a 
> >>directory-oriented presentation of the kernel memory and 
> >>grows as needed.  The system creates it on the fly.
> > 
> > 
> > Where is the data stored?  It's not all in memory.  If it 
> shows 919MB, 
> > I assume it's gotta be juggling that somewhere.
> 
> It's in RAM and munged by the proc filesystem.  All you need 
> is the /proc mountpoint.  How did you get 919MB anyway?  "df 
> -h" should show something like:
> 
> 	# df -h /proc
> 	Filesystem            Size  Used Avail Use% Mounted on
> 	none                     0     0     0   -  /proc
> 
> If it shows something different, I'd sure like to know what 
> the devil it is.


Hmm.. I get the same as you get using df, when I got the 919MB, I used
du yesterday, but now I'm getting "0" when I do a du.  No idea what
happened, the exact command line I used was "du -sk /proc".