Filesystem fragmentation and scatter-gather DMA

[Ric Wheeler]

Jon Forrest wrote:
> Ric Wheeler wrote:
> 
>> Every level of the the system tries to guess how to combine and read 
>> ahead, all the way from the file system down to the internal firmware 
>> in  the storage.
> 
> I remember Kirk McKusick once complaining about how hard it was to write
> a file system when so many other levels in a system try to second guess
> what he was trying to do. I've also heard disk engineers complain about
> the same thing, except they complain about the OS people not leaving
> optimization techniques to them. Go figure.

The trick is just to do the obvious thing (big, sequential IO's) from 
the application to give the various layers the easiest job of second 
guessing ;-)

There are certainly advantages to doing the read ahead (and coalescing) 
at the different layers. For example, a file system can do predictive 
read ahead across the non-contiguous chunks of a single file while the 
IO layer can coalesce multiple write or read commands on the same host 
and a multi-ported drive can do the same for multiple hosts.


> 
>> I think that fragmentation is a bad performance hit, but that we 
>> actually do relatively well in keeping our files contiguous in normal 
>> cases.
> 
> We might disagree on how bad the performance hit is, but I'm really
> trying to prevent non-technical people from panicking when they see
> a fragmented filesystem (or file).

I agree - most casual users will never see anything close to a 
performance issue until they have completely filled the file system. In 
that case, defragmentation will not be the real help.

> 
>> I have a simple bit of c code that uses fibmap to dump the 
>> sectors/blocks for a specific file. If you like, I can send it over to 
>> you.
> 
> Sure. Thanks.

I will send it to you out of band. Mark Lord had some tweaks to this 
that I have not rolled in, let me know if it is useful.

ric