Should SQLite users be setting barrier=1?

[Ted Ts'o]

On Tue, Jul 13, 2010 at 11:56:24PM +0700, Dan Kennedy wrote:
> Richard put a paragraph with a link to your answer in
> our documentation here:
> 
>   http://www.sqlite.org/draft/lockingv3.html#ext3-barrier-problem
> 
> Please let us know if this misrepresents the situation.
> Or if there is something else we should add to clarify
> it.

Something else that might be useful to put in the FAQ is some
suggestions to application programmers about methods to get no more
than the safety needed for their application.  For example, firefox's
"awesome bar" (myself, I don't think it's so awesome) at one point was
doing an SQL COMMIT after every single time a user visited a page
(either by clicking on a link or entering a URL in the "awesome bar").
Worse yet, they were doing this in the main UI loop of their
application.  Not only was this a performance disaster, if you had an
SSD it was doing you no favors, the firefox/SQLite combination was
also doing a third of a megabyte of disk writes for every single page
that you visited.

Did they really need that level of safety?  Probably not.  It probably
would have been better if they had used a memory-only SQLite database
for immediate history, and then every 50 pages or so, in a background
thread, contents of the in-memory SQLite database could be flushed to
the disk-resident SQLite database.  After all, if someone exits a 3D
game and their crappy proprietary ATI or Nvidia driver crashes their
laptop, remember the last or 10 pages they web browser history might
not be the most important thing in the world....

So putting in some text to help application programmers think about
performance issues as well as robustness issues, and creative ways of
trading off between them, I think would be a good idea.  It's a rare
application programmer these days who think at the XML and HTML and
Java/Python level, and then understand what SQLite is doing, and then
understand the implications at the level of the kernel, the hard disk,
SSD, write wear issues, and barriers.

Thanks, regards,

						- Ted