[linux-lvm] LVM limits? OT
Steeve McCauley
steeve at terrascale.net
Fri Feb 1 13:08:00 UTC 2008
David Robinson wrote:
> Jordi Prats wrote:
>> Hi,
>> I'm the system administrator of PADICAT (http://www.padi.cat). It
>> collects Catalan (http://en.wikipedia.org/wiki/Catalonia) web sites to
>> provide permanent access to them (http://www.padi.cat/en/quees.php).
>> It's equivalent to Internet Archive (http://www.archive.org) but for a
>> particular culture.
>>
>> Our software developers require us to have one large file system,
>> actually a single directory, with all this historically-classified web
>> sites on a gziped file.
>>
>> I'm currently studying lustre and other HPC-related file systems to get
>> this large file system, but by now I have ext3 as our file system. Next
>> Monday I'm planning to extend it to 3TB o 4TB, so I'm currently
>> researching for restrictions because during next month I'll have between
>> 3TB to 4TB more to add: so, it will become a 8TB file system.
>>
>> Last time I fsck my 2'1TB file system I spend about 2 hours. Anyway, I'm
>> also curious about the maximums :P
>
> The man page for vgcreate talks a little bit about limits:
>
> "If the volume group metadata uses lvm1 format, extents can vary in size
> from 8KB to 16GB and there is a limit of 65534 extents in each logical
> volume. The default of 4 MB leads to a maximum logical volume size of
> around 256GB.
> If the volume group metadata uses lvm2 format those restrictions do not
> apply, but having a large number of extents will slow down the tools but
> have no impact on I/O performance to the logical volume."
>
> In short, you're more likely to reach filesystem limits before LVM's.
> EXT3 has a theoretical limit of 32 TB, but 32 GB its not practical.
> Creating an EXT3 filesystem larger than 8 TB is umm, brave - as you have
> noticed the tools (eg. fsck) do not scale well w/ EXT3.
With a 4K block size the theoretical limit is 16TB. The ext2/3
tools do not work beyond 4TB, or at least they didn't as of a year
or two ago. I did report the problem to Theodore, not sure if the
fix was ever adopted, but you're absolutely right about the scalability
problems of ext3. An fsck of a 16TB filesystem was something that
definitely required tea.
> GFS or XFS (or others) may be more suitable, but it depends on your
> requirements.
Any of the extent based filesystems. A fixed format filesystem like
ext3 does not scale.
More information about the linux-lvm
mailing list