Current state of conventional file systems on Flash & Linux....

[Jim Gettys]

I had a conversation with Dave Woodhouse this morning, that people
should internalize.

The wonderful demoing of running Fedora, Debian, Ubuntu on a SD card,
without thought about where what is stored, is something likely to come
back to bite us downstream as people use it as a serious support
problem.  There is real danger, particularly on commodity media, of
wearing out the flash device.

We need to give serious thought to where what gets stored, on what file
system.  The only wear leveling we can *count* on is on JFFS2 on our
internal flash.  And we can strongly encourage the use of particularly
"good" vendors/models of flash.
                             - Jim


<jg> If you are running a conventional file system on a flash device,
what's the right solution for wear leveling right now for conventional
SD or USB.
<dwmw2> Prayer. .
<dwmw2> SD and USB stuff -- none of the things with the built-in
translation layers -- have ever been observed to be reliable enough to
contemplate using them.
<dwmw2> although hopefully the new set of SSD stuff for laptops will
mean they have to improve.
<dwmw2> but you can't really know what's going on under the covers.
<jg> yeah, I know I can't rely on the hardware.
<dwmw2> They might be doing some form of wear leveling; they might not
<jg> the question is, do we have any wear leveling software layer?
<dwmw2> often, if the devices do, they'll do it within certain regions.
Not really for wear leveling on a block device. We have stuff like FTL
which lets use emulate a block device on top of real flash and that
obviously does wear leveling, but we don't have anything which will
deliberately spread writes around on a block device.
That probably would want to live in the file system itself, rather than
a block<->block mapping layer -- although I dare say you _could_ do it
with some kind of DM plugin; it's probably something we should look at
doing in btrfs. Btrfs has a 'wandering tree' setup, a bit like logfs.
I'm planning to see if I can make it work on raw flash, too. Should be
really good.
<jg> ok, for now,  you *really* want to be careful on what you put where
you put what, and what kind of media you use if you put a conventional
system, say, on SD, to get the best you can get on wear characteristics.
<dwmw2> yeah. I wouldn't want to bet the farm on SD or _anything_ which
does internal stuff for you. I've never met one that I'd want to take
home and meet my mother.
Would be an interesting project to tweak block allocation schemes in
traditional file systems to think about wear leveling and not
defragmentation.
<jg> That's what I thought; probably why our M$ friends are being so
picky on which SD card they use. 
<dwmw2>that and the speed of SD cards of course -- they don't want slow
ones :)
<jg> yup.
<jg> Thanks for the insight; it is roughly what I thought, but as I
don't track the area, I wanted to be up-to-date....
<dwmw2> Nothing much has changed since we dropped the magic translation
layer chip from the XO when we first started
<jg> ah, but for our internal flash, we have jffs2.
<dwmw2> the trend of putting SSDs in laptops will have a decent effect
in time, I hope, but it hasn't yet. Jffs2 doesn't scale much further. We
need ubifs/logfs or maybe btrfs in future. The not-so-distant future, I
hope. I'm allegedly going to be getting a shiny new translation layer
(make flash look like disk) soon, which I'm supposed to make look like
proper Linux code and merge upstream


-- 
Jim Gettys <jg at laptop.org>
One Laptop Per Child