[augeas-devel] libguestfs crash in libaugeas

[David Lutterkort]

On Tue, 2010-08-03 at 20:11 +0100, Richard W.M. Jones wrote:
> On Tue, Aug 03, 2010 at 11:11:57AM -0700, David Lutterkort wrote:
> > On Tue, 2010-08-03 at 11:54 +0100, Matthew Booth wrote:
> > > We're having a problem with libguestfs's daemon crashing during v2v. The 
> > > crash is in libaugeas. We unfortunately don't have a small reproducer, 
> > > but we have finally managed to get a core dump. The stack trace looks 
> > > like this:
> > > 
> > > #0  0x00007fb490f3763b in pathx_symtab_remove_descendants (
> > >      symtab=<value optimized out>, tree=0x5941900) at pathx.c:2569
> > > #1  0x00007fb490f3580b in tree_rm (p=0x8177da0) at augeas.c:890
> > > #2  0x00007fb490f363e5 in aug_rm (aug=0x57b5e20, path=<value optimized out>)
> > >      at augeas.c:911
> > > #3  0x00007fb490f36b08 in aug_save (aug=0x57b5e20) at augeas.c:1192
> > > #4  0x0000000000403992 in do_aug_save () at augeas.c:332
> > > #5  0x000000000042194f in aug_save_stub (xdr_in=<value optimized out>)
> > >      at stubs.c:623
> > > 
> > > Interesting local variables:
> > > 
> > > (gdb) print t
> > > $4 = (struct tree *) 0xc1
> > > (gdb) print *tree
> > > $5 = {next = 0x0, parent = 0x66a5130, label = 0x64f55a0 "saved",
> > >    children = 0x0, value = 0x6a32290 "/files/boot/grub/menu.lst", dirty = 0}
> > > 
> > > This is augeas-libs-0.7.2-3.fc13.x86_64.
> > 
> > Looks like memory is getting corrupted before things blow up here. More
> > specifically, the nodeset for a variable has been corrupted. What's in
> > tab->name ? (That's the name of the variable)
> > 
> > > There are many circumstances in which this conversion works. For 
> > > example, if I run the conversion again against the partially converted 
> > > guest it will succeed. This leads me to believe that the problem is 
> > > unlikely to be local to the crash.
> > > 
> > > Any ideas? I can provide the crashed daemon and core dump for inspection 
> > > if required. I'm also more than happy to test augeas patches.
> > 
> > The following would be really useful in putting together a reproducer
> > with augtool, not sure how realistic getting that is:
> > 
> >       * A list of the augeas commands that have been executed since the
> >         last call to aug_init up to the point of the crash
> >       * A tarball of /etc of the image that causes the crash
> >       * A list of the defined variables if we can't get a list of
> >         commands - you can get them with 'aug_print(aug,
> >         "/augeas/variables")' just before the aug_save call that blows
> >         up.
> 
> David, you mentioned in another email that there is a variable "new".
> 
> Indeed, virt-v2v defines this:
> 
>         $g->aug_defnode('new',
>                         '/files/boot/grub/menu.lst/title[last()+1]',
>                         $title);
> 
> from:
> http://git.fedorahosted.org/git/?p=virt-v2v.git;a=blob;f=lib/Sys/VirtV2V/GuestOS/RedHat.pm;h=101a64b63b38c5e965a0e5ae34404679de059403;hb=HEAD#l975
> 
> I hope Matt is able to provide the other details you requested
> tomorrow.

Hopefully - I tried to simulate some of what that perl module is doing
(based on my reading of the code :( ), and ran the following through
augtool:

        set /augeas/save backup
        load
        
        defvar template /files/boot/grub/menu.lst/title[1]
        
        rm /files/etc/modules.conf/alias[102]
        rm /files/etc/modules.conf/alias[118]
        rm /files/etc/modules.conf/alias[200]
        
        match /files/boot/grub/menu.lst/title/kernel
        defnode new /files/boot/grub/menu.lst/title[last()+1] "New
        stanza"
        set $new/root "(hd0,0)"
        set $new/kernel "/vmlinuz-2.6.24.4-64.fc8"
        set $new/initrd "/initrd-2.6.24.4-64.fc8.img"
        clear $new/kernel/ro
        set $new/kernel/root /dev/vg00/lv00
        
        match /files/boot/grub/menu.lst/title
        
        match $new
        
        set /files/boot/grub/menu.lst/default 3
        save

That all works as expected, and doesn't raise anything suspicious under
valgrind. Hopefully, a complete list of augeas commands that v2v runs
will shed some light on what's going wrong.

David