[libvirt] PATCH: Disable QEMU drive caching
Daniel P. Berrange
berrange at redhat.com
Wed Oct 8 16:33:37 UTC 2008
On Wed, Oct 08, 2008 at 11:06:27AM -0500, Anthony Liguori wrote:
> Daniel P. Berrange wrote:
> >On Wed, Oct 08, 2008 at 01:15:46PM +0200, Chris Lalancette wrote:
> >>Daniel P. Berrange wrote:
> >>>QEMU defaults to allowing the host OS to cache all disk I/O. THis has a
> >>>couple of problems
> >>>
> >>> - It is a waste of memory because the guest already caches I/O ops
> >>> - It is unsafe on host OS crash - all unflushed guest I/O will be
> >>> lost, and there's no ordering guarentees, so metadata updates could
> >>> be flushe to disk, while the journal updates were not. Say goodbye
> >>> to your filesystem.
> >>> - It makes benchmarking more or less impossible / worthless because
> >>> what the benchmark things are disk writes just sit around in memory
> >>> so guest disk performance appears to exceed host diskperformance.
> >>>
> >>>This patch disables caching on all QEMU guests. NB, Xen has long done
> >>>this
> >>>for both PV & HVM guests - QEMU only gained this ability when -drive was
> >>>introduced, and sadly kept the default to unsafe cache=on settings.
> >>I'm for this in general, but I'm a little worried about the "performance
> >>regression" aspect of this. People are going to upgrade to 0.4.7 (or
> >>whatever),
> >>and suddenly find that their KVM guests perform much more slowly. This is
> >>better in the end for their data, but we might hear large complaints
> >>about it.
> >
> >Yes & no. They will find their guests perform more consistently. With the
> >current system their guests will perform very erratically depending on
> >memory & I/O pressure on the host. If the host I/O cache is empty & has
> >no I/O load, current guests will be "fast",
>
> They will perform marginally better than if cache=off. This is the
> Linux host knows more about the underlying hardware than the guest and
> is able to do smarter read-ahead. When using cache=off, the host cannot
> perform any sort of read-ahead.
>
> >but if host I/O cache is full
> >and they do something which requires more host memory (eg start up another
> >guest), then all existing guests get their I/O performance trashed as the
> >I/O cache has to be flushed out, and future I/O is unable to be cached.
>
> This is not accurate. Dirty pages in the host page cache are not
> reclaimable until they're written to disk. If you're in a seriously low
> memory situation, they the thing allocating memory is going to sleep
> until the data is written to disk. If an existing guest is trying to do
> I/O, then what things will degenerate to is basically cache=off since
> the guest must wait for other pending IO to complete
>
> >Xen went through this same change and there were not any serious
> >complaints, particularly when explained that previous system had
> >zero data integrity guarentees. The current system merely provides an
> >illusion of performance - any attempt to show that performance has
> >decreased is impossible because any attempt to run benchmarks with
> >existing caching just results in meaningless garbage.
> >
> >https://bugzilla.redhat.com/show_bug.cgi?id=444047
>
> I can't see this bug, but a quick grep of ioemu in xen-unstable for
> O_DIRECT reveals that they are not in fact using O_DIRECT.
Sorry, it was mistakenly private - fixed now.
Xen does use O_DIRECT for paravirt driver case - blktap is using the combo
of AIO+O_DIRECT. QEMU code is only used for the IDE emulation case which
isn't interesting from a performance POV.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
More information about the libvir-list
mailing list