How to generate a large file allocating space

[Alex Bligh]

Ted,

--On 4 November 2010 15:17:34 -0400 Ted Ts'o <tytso at mit.edu> wrote:

> Patches to do this wouldn't be that hard.  The harder part would
> probably be the politics on fs-devel regarding the semantics of
> FALLOC_FL_EXPOSE_OLD_DATA.

Also presumably there would be some pressure to make it work for
every filesystem that supported fallocate().

>> In the mean time, particularly as I'd ideally like to avoid a kernel
>> modification, is there a safe way I could use or modify the ext2
>> library to run through the extents of a fallocated() file and clear
>> the "unwritten" bit? If I clear that (which from memory is the top
>> bit of the extent length), is that alone safe? (on an unmounted
>> file system, obviously).
>
> Yes, there most certainly is.  The functions you'd probably want to
> use are ext2fs_extent_open(), and then either use ext2fs_extent_goto()
> to go to a specific extent, use ext2fs_extent_get() with the
> EXT2_EXTENT_NEXT operation to iterate over the extents, and then use
> ext2fs_extent_replace() to mutate the extent.  Oh, and then use
> ext2fs_extent_close() when you're done looking at and/or changing the
> extents of a file.
>
> If you build tst_extents in lib/ext2fs, you can use commands like
> "inode" (to open the extents for a particular inode), and "root",
> "current", "next", "prev", "next_leaf", "prev_leaf", "next_sibling",
> "prev_sibling", "delete_node", "insert_node", "replace_node",
> "split_node", "print_all", "goto", etc.  Please don't use this in
> production, but it's not a bad way to play with an extent tree, either
> for learning purposes or to create test cases.  tst_extents.c is also
> a good way of seeing how the various libext2fs extent API's work.

Thaks, that's really helpful. Are the extents always the leaves? IE
will next_leaf take me through extent by extent?

Does your "please don't use this in production" warning apply to
tst_extents.c or to the whole of lib/ext2fs? The library calls
seem quite a good way to get the list of extents and are
presumably what fsck etc. use.

> Well, things like Hadoopfs exist today, and Ceph (if you need a
> POSIX-level access)

No, just block layer access fortunately

> is admittedly less stable.  But if you're starting
> from scratch, wouldn't that be pretty far away from readiness as well?

The idea was to base as much as possible on existing running code (e.g.
ext4) with as few variations as possible. I'd be very surprised if we
end up exceeding a few thousand lines of code. All the cluster, lock
management etc we are borrowing from elsewhere, for instance.

-- 
Alex Bligh