Things to do this week instead of arguing about mixers

[Eric Sandeen]

Michael Cronenworth wrote:
> -------- Original Message --------
> Subject: Re: Things to do this week instead of arguing about mixers
> From: Eric Sandeen <sandeen at redhat.com>
> To: Development discussions related to Fedora <fedora-devel-list at redhat.com>
> Date: 04/27/2009 03:53 PM
> 
>> Try downloading a large (multi-gigabyte) torrent on ext4 or xfs or
>> btrfs; see what the resulting file layout looks like if your client does
>> not preallocate.  Now use a client that preallocates the space and try
>> again.  Calling fallocate() first for the full file size will help on
>> all these filesystems (well, TBH, I have not tested it on btrfs).
>>
>> If there are well-defined interfaces to give hints to the filesystem
>> about the ultimate state of a file, it's a nice feature, not a bug.
>>
>> Sure, filesystems should do the best they can in the absence of hints,
>> but performing well (or better) _with_ a hint is not indicative of a bug.
>>
> 
> I can truely only speak for XFS, but btrfs might be similar:
> 
> XFS waits[1] to allocate space for a few seconds in order to see the 
> full file size of the write() you want to commit. This allows for a more 
> complete allocation at the cost of data integrity.

Thanks, I'm familiar with xfs.  Ext4 also implements delayed allocation.

And there is no cost to data integrity, really, if the opposite of
integrity is corruption.  What you're probably referring to is the lack
of ext3's 5s journal commit + data=ordered flush, but let's PLEASE not
rehash that thread.  ;)

But anyway, unless you can download your entire torrent within that
delalloc window, you may not benefit a whole lot from it.

Just for fun I downloaded the snap1 live dvd via deluge (which uses
rb_libtorrent for the backend) onto xfs using "full allocation" (which I
think really just means "fill the file in piecemeal as it arrives) and
got 2601 discontiguous extents... on ext4 I got 32 extents.  This is
actually indicative of some different ext4 allocator behavior, which
physical block gaps for logical block gaps; it may not be ideal in all
cases, but works pretty well in this one.  ;)

> deluge, one bittorrent client, gives you the option of full or dynamic 
> allocation. I download multi-gigabyte torrents often with deluge and 
> full allocation.
> 
> In the end -- SSDs should be cheaper so this discussion is moot.

Until the last spindle stops spinning, I'm not so sure.  :)  And FWIW,
while there may not be a seek penalty, highly fragmented files may still
cause you to do a lot of small IOs even on an ssd, so it's not
completely a non-issue.

> I apologize. I'm a rambling idiot. Please carry on without me.

Nah, no worries.  There is an awful lot of (sometimes mis-)information
out there, it's hard to keep track of it all, and good to have useful
discussions, I think.  :)

Maybe you can file an RFE against rb_libtorrent? :)

-Eric

> [1] http://en.wikipedia.org/wiki/Xfs#Delayed_allocation
>