[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Question about file system capacity



On Aug 30, 2001  13:31 -0700, Lee Go-Tze wrote:
> pardon my ignorance on ext2/3fs.  Is the ext2fs' 4Tb file system
> and 2GB maximum file size limit still true in ext3fs? 

There are no such limits on the filesystem itself.  See
linux/Documentation/filesystems/ext2.txt for more information.

The 2GB file limit was fixed in 2.2 if you have LFS kernel patches,
and is part of stock 2.4 kernels (this limit was only in the ext2 kernel
code/VFS and not in the ext2 format itself).  Of course you need a user
environment which can support large files (libc, cat, dd, etc.)

There is a Linux VFS limit of 2TB (or maybe 1TB depending on drivers)
for a single block device (including LVM and MD RAID).  The 2TB block
device limitation will be fixed in 2.5, at which point the _current_
limits for ext2/ext3 depend on the block size chosen.  You almost
always want to have a 4kB block filesystem, which gives you 16TB maximum
(Alphas can use 8kB block filesystems with 32TB max).

No doubt when we start hitting the  limits (for any reasonable number
of users) there will be a compatible change to the ext2 disk layout
(such as extent-based allocation) to fix this.

<thinking cap on>
Maybe, for allocation of large files, we could do allocations of whole
block groups at a time?  For 4kB blocks, this would give us up to
128MB "megablock" sizes.  It would save having to handle thousands of
bits in bitmaps, and we could work out a scheme to do "tail packing"
with regular blocks.

To make this work, we should keep some number of groups free of inode
tables and assigned blocks/inodes, only doing inode/block allocation in
earlier groups.

Alternately, we could carve these block groups up into extents, and
assign those to files.  We are nowhere close to using up group numbers
(2^32 * 128MB = 512PB), so it is purely the 2^32 limit on block numbers
that is hurting us.
</thinking cap off>

Ted, do you have a URL to your "extents for ext2" paper available?

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]