Disk partitions and LVM limits - SUMMARY

[Peter Blajev]

Thank you all for the help.
I'm writing this summary message because of people requests. I haven't tried 
all of this. I just collected it and organized it.

You've got a big storage. Now what?
The short answer is: "Just connect it. It should work."

I'll play safe by saying that the following applies to <10TB storage. Some 
people reported file systems of 80TB.

Things to watch out for:
  - Make sure the driver you are using or the storage itself don't restrict 
you from making big partitions or file systems.
  - fdisk creates partitions up to 2.1TB in size. Use "parted" instead.
  - RHEL5 supports up to 8TB ext3 file system. To create bigger then 8TB use 
option "-F" and 4K blocks.

Your options are:
  - If the storage is connected to a RAID controller you can use the 
controller to create smaller logical partitions. Then combine them with LVM.

 - If you really want to partition the drive use parted.
   While partitioning if you run into this message:
===
sdb: very big device. try to use READ CAPACITY(16).
SCSI device sdb: 10248519680 512-byte hdwr sectors (5247242 MB)
sdb: Write Protect is off
===
    you can just ignore it. It's informational (based on a few people 
replies).
    If you don't like parted you can still use fdisk to create a few 2TB 
partitions and then use LVM.

  - Use LVM on the raw device. Don't partition.
    This is what I did and it worked for me.
    Make sure you wipe out the MBR first:
    dd if=/dev/zero of=/dev/sdX bs=512 count=63
      (count=1 should work too but 63 won't hurt either)

    Create the pv, vg and lv:
      pvcreate /dev/sdX
      vgcreate <vg name> /dev/sdX
      lvcreate -L <size> -n <name> -v vg_name

Now you have to create the file systems. Some people recommend XFS.
If you try to create bigger then 8TB ext3 file system make sure you use
option "-F" and 4K blocks:
     mkfs -t ext3 -F -b 4096 /dev/vg00/lvol01

For more information:
  http://www.centos.org/product.html
  http://kbase.redhat.com/faq/FAQ_103_11461.shtm
  The email thread "Disk partitions and LVM limits" in CentOS, RedHat and
     RedHat-Sysadmin mailing lists.

Thank you
Peter

> Hi,
>
> I've got a DAS DELL MD1000 with a bunch of SATA drives in RAID 5
> configuration with total space of 5.4TB. This box is attached to a
> CentOS5 system (kernel 2.6.18-53.1.6.el5).
>
> Any idea how to make this space usable?
> Is there a limit how big a partition can be? What is the work around?
> Is there a limit how big a file system ca be?
>
> I've tried to partition it but no matter how bug partition I create
> fdisk spits out these messages on the console:
> ---
> sdb: very big device. try to use READ CAPACITY(16).
> SCSI device sdb: 10248519680 512-byte hdwr sectors (5247242 MB)
> sdb: Write Protect is off
> ---
>
> I decided to not partition the drive and use LVM but the physical volume
> stopped at 2TB.
>
> So, right now I can't use LVM because of this 2TB limit and I'm not sure
> if I partition the drive how good these partitions are because of the
> the message from fdisk.
>
> Any help or idea is highly appreciated.
>
> Thank you
> Peter