[Libguestfs] running libguestfs as a service?

[Richard W.M. Jones]

On Fri, Apr 29, 2011 at 01:38:19AM -0400, T Johnson wrote:
> Hello,
> 
> First off, many thanks for libguestfs... very useful!
> 
> I'm trying to run libguestfs as an RPC service by using a python
> daemon with the python libguestfs bindings. With this service I'm
> "centrally" editing VM images, and often running many GuestFS() on the
> same image file in a short amount of time to edit the image's
> contents.
> 
> The problem I'm running across is that each GuestFS() instance called
> from within the python daemon doesn't terminate itself (until a
> timeout much later?). I believe this sometimes causes my image edits
> to clobber each other and corrupt the image now and then. Each image
> edit usually amounts to:
> 
> g = guestfs.GuestFS()
> g.add_drive(image_file)
> g.launch()
> g.mount("/dev/vda1", "/")
> g.write(file_name, file_contents)
> g.sync()
> g.umount_all()
> g.close()
> del g
> 
> 
> As I understand things, there is a reference count that should close
> "g" once the reference count = 0. However, I can't seem to make that
> happen inside of my python daemon which continues to run.

First thing is to find out if 'del g' really closes the handle or not.
It's my understanding (of Python bindings) that 'del g' *should* close
the handle immediately, assuming you haven't hidden a reference to 'g'
somewhere else in the program.

Unfortunately we left out a trace call in guestfs_close so instead of
using ordinary call tracing you have to do this using debugging:

  g.set_verbose (1)
  print >> sys.stderr, "I'm about to call del g ..."
  del g
  print >> sys.stderr, "I've called del g."

This should produce a sequence of messages like:

  I'm about to call del g ...
  closing guestfs handle 0x1234 (state 2)  # ie. guestfs_close called
  I've called del g.

So we could get one of two results here: either guestfs_close is being
called, or guestfs_close is not being called.  If it's *not* being
called then it's something to do with reference counting (or you're
tucking a reference away somewhere else).  I don't know a lot about
how Python refcounting is supposed to work, but it's possible there is
a mistake in the bindings, eg. not incrementing or decrementing a
reference somewhere.

If guestfs_close *is* being called then it's a more serious problem in
the underlying library.  guestfs_close is supposed to kill the qemu
subprocess by sending it a SIGTERM, so really it should exit almost
immediately, and we also waitpid for the process so really
guestfs_close should not be returning before qemu has really exited.
If this isn't happening, then getting 'strace' output would be the
most useful thing.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/