[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: How to generate a large file allocating space



Ted,

--On 4 November 2010 15:17:34 -0400 Ted Ts'o <tytso mit edu> wrote:

Patches to do this wouldn't be that hard.  The harder part would
probably be the politics on fs-devel regarding the semantics of
FALLOC_FL_EXPOSE_OLD_DATA.

Also presumably there would be some pressure to make it work for
every filesystem that supported fallocate().

In the mean time, particularly as I'd ideally like to avoid a kernel
modification, is there a safe way I could use or modify the ext2
library to run through the extents of a fallocated() file and clear
the "unwritten" bit? If I clear that (which from memory is the top
bit of the extent length), is that alone safe? (on an unmounted
file system, obviously).

Yes, there most certainly is.  The functions you'd probably want to
use are ext2fs_extent_open(), and then either use ext2fs_extent_goto()
to go to a specific extent, use ext2fs_extent_get() with the
EXT2_EXTENT_NEXT operation to iterate over the extents, and then use
ext2fs_extent_replace() to mutate the extent.  Oh, and then use
ext2fs_extent_close() when you're done looking at and/or changing the
extents of a file.

If you build tst_extents in lib/ext2fs, you can use commands like
"inode" (to open the extents for a particular inode), and "root",
"current", "next", "prev", "next_leaf", "prev_leaf", "next_sibling",
"prev_sibling", "delete_node", "insert_node", "replace_node",
"split_node", "print_all", "goto", etc.  Please don't use this in
production, but it's not a bad way to play with an extent tree, either
for learning purposes or to create test cases.  tst_extents.c is also
a good way of seeing how the various libext2fs extent API's work.

Thaks, that's really helpful. Are the extents always the leaves? IE
will next_leaf take me through extent by extent?

Does your "please don't use this in production" warning apply to
tst_extents.c or to the whole of lib/ext2fs? The library calls
seem quite a good way to get the list of extents and are
presumably what fsck etc. use.

Well, things like Hadoopfs exist today, and Ceph (if you need a
POSIX-level access)

No, just block layer access fortunately

is admittedly less stable.  But if you're starting
from scratch, wouldn't that be pretty far away from readiness as well?

The idea was to base as much as possible on existing running code (e.g.
ext4) with as few variations as possible. I'd be very surprised if we
end up exceeding a few thousand lines of code. All the cluster, lock
management etc we are borrowing from elsewhere, for instance.

--
Alex Bligh


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]