[Libguestfs] getting guestfs_rsync_out to work

[Richard W.M. Jones]

On Mon, Feb 09, 2015 at 08:01:31AM -0600, Jeff Brower wrote:
> Richard-
> 
> > On Sun, Feb 08, 2015 at 12:11:37PM -0600, Jeff Brower wrote:
> >> With continuous loop testing, what we found is that we have to shut
> >> down and re-launch the image handle to see changes on the Win7 live
> >> guest.  Unfortunately image re-launch takes time, 3-5 sec (the image
> >> size is 50 GByte).  I'm assuming this is because libguestfs makes an
> >> internal copy of filesystem, and works from that, and doesn't
> >> "refresh" this internal copy until re-launching the image handle.
> >
> > This isn't surprising.  Libguestfs doesn't make an internal copy of
> > the whole filesystem, but the libguestfs appliance will have a copy
> > (in its kernel memory) of any parts of the disk that rsync read.
> >
> > See also:
> > http://libguestfs.org/guestfs.3.html#architecture
> >
> >> Could we create a second partition on the guest that is much
> >> smaller, say 50 MByte, attach the image to that, and thus reduce the
> >> shutdown and re-launch time into the msec range?  Is there another
> >> real-time method, not using rsync?
> >
> > The time to relaunch the appliance doesn't depend on the size of the
> > disk.
> >
> > You could try hotplugging but it may not be much faster:
> >
> > http://libguestfs.org/guestfs.3.html#hotplugging
> >
> > I think what you really need to do is to install a backup agent in the
> > Windows guest and use a regular network backup.
> 
> We don't need to use rsync.  We know which files will change.  For example even if we do this (error checking removed):
> 
>    while (1) {
> 
>       guestfs_mount_options(g, "sync", "/dev/sda2", "/");
> 
>       printf("%s\n", guestfs_cat(g, "/HostShared/temp.txt"));
> 
>       guestfs_umount(g, "/");
> 
>       guestfs_fsync(g);
> 
>       usleep(1000*1000);
>    }
>
> temp.txt from the Win7 guest is copied correctly the first time, but
> afterwards any change we make to it (e.g. using Win7 Notepad) is
> never seen.  That's the issue.

This is completely expected.  Have a look at the architecture diagram
here:

http://libguestfs.org/guestfs.3.html#architecture

Now imagine there is another line drawn from the "Device or disk
image" box to your running Windows guest.  How does the appliance know
that there's something else making changes to the disk?

BTW you really *must* be using the readonly flag when adding disks
(guestfs_add_drive_opts) else you will get disk corruption.

> Is there any way at all to accomplish this using libguestfs, without
> re-launching the image, which is time-consuming?

You could try calling `guestfs_drop_caches (g, 3);' (which may or may
not work depending on a bunch of details in how Linux and qemu works),
or using hotplugging.

However you need to understand how libguestfs, the kernel and qemu are
architected and why the naive approach is failing first.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org