[libvirt] RFC: Building a virtlogd daemon

[Daniel P. Berrange]

With QEMU there are a couple of areas where QEMU ends up logging data
to a file on the host.

 - stdout/err - connected to /var/log/libvirt/qemu/$GUEST.log
 - serial/parallel/console when used with type=file

The stdout/err is typically very small, typically only getting data when
an error occurs in QEMU resulting in it existing. In the past there was
the chance of it getting lots of data - eg spice used to write tonnes of
data on stderr during normal operation, so a guest OS could cause large
data to be saved to the host log file. This is harder todo now, but still
theoretically possible.

The serial/parallel/console log files can be arbitrarily sized and under
complete control of the guest OS - it can write whatever it wants with
no limits.

In both these cases we would prefer to be able to limit the amount of
data a guest can write to the host file, so we have a finite cap on
disk usage. You might suggest using logrotate, but that only runs
periodically so between invocations of logrotate the log files can
still grow to arbitrary size. This is inherant problem in doing the
log rotation out of band / asynchronously

OpenStack has a further problem it would like to solve. It wants to
record all guest serial port data to a log file, as it has an API for
a client user to read historically captured data from the serial port.
This is done using type=file chardev

At the same time it wants to expose an interactive console for client
users to interact with the guest. This is done using type=unix or
type=pty chardevs.

The obvious problem is that a single serial/parallel/console device
can only be connected to one chardev backend at a time. A very long
time ago (5-6 years?) I submitted patches to allow multiple chardev
backends in QEMU but they were rejected.

Downstream projects like OpenStack have considered setting up console
log handling service themselves, but my view is that this is a problem
that all users of libvirt face, so libvirt should provide a standard
solution to it. This avoids each downstream app reinventing the wheel.

So I'm intending to create a standalone virtlogd daemon to address this
problem. Similarly to virtlockd, it will be able to re-exec itelf so
that upgrades can be done with no interruption to logging, and libvirtd
will talk to it over a simple RPC protocol.

 - For stdout/stderr

    - libvirtd will ask virtlogd will provide a pipe FD that can
      be connected to the guest stdout/err, in the sme way libvirtd
      provides a file FD today.

    - virtlogd will read from this pipe and write the data to
      /var/log/libvirt/qemu/$GUEST.log

    - virtlogd will either truncate or rotate the $GUEST.log
      file when it reaches a declared certain size limit.

 - For serial/parallel/console

    - libvirtd will ask virtlogd to setup a pair of UNIX domain
      sockets listening on

          /var/run/virtlogd/qemu/$GUEST.guest
          /var/run/virtlogd/qemu/$GUEST.client

    - libvirtd will ask virtlogd to (optionally) write the data
      to a file /var/log/virtlogd/qemu/$GUEST.log

      (Or /var/log/libvirt/qemu/$GUEST-$DEVICE.log perhaps,
       where $DEVICE is the <alias> string from the console,
       serial or parallel device ?)

    - QEMU will be told to connect to this $GUEST.guest socket
      with the type=unix chardev backend

    - The virDomainOpenConsole API will connect to the
      $GUEST.client socket

    - virtlogd will read data from $GUEST.guest socket and
      write it to $GUEST.client and optionally $GUEST.log too

    - virtlogd will read data from $GUEST.client socket and
      write it to $GUEST.guest socket

    - virtlogd will either truncate or rotate the $GUEST.log
      file when it reaches a declared certain size limit.

So at the end of this we will have strictly size limited log files which
can rotated at the size threshold, and the ability to have interactive
console sessions and file logging at the same time.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|