[Libguestfs] [PATCH 2/2] actions: refactor available & feature_available

Fri Nov 6 22:07:20 UTC 2015

On Fri, Nov 06, 2015 at 09:40:53PM +0000, Richard W.M. Jones wrote:
> On Thu, Nov 05, 2015 at 04:56:43PM +0100, Pino Toscano wrote:
> > Refactor the internal_feature_available to return the result for just
> > one group, so it is easier to know on the library side what was the
> > actual error, and which group refers to; drop internal_available, as no
> > more needed after this.
> > 
> > On the library side, implement in available and feature_available the
> > real logic to iterate through the requested group, and error out or
> > return whether the groups are available. This also introduces caching
> > for the features, so each needs to be queried just once in each
> > appliance run.
> > 
> > The result of this is that there should be much less communication with
> > the daemon to know about available features; the downside is that
> > queries for more than one group at once, not already cached, will be
> > cause more queries to the daemon.
> 
> This commit causes the following valgrind failures.  Curiously they
> only occur in virt-v2v:
> 
> ==25905== HEAP SUMMARY:
> ==25905==     in use at exit: 3,604,364 bytes in 1,663 blocks
> ==25905==   total heap usage: 83,930 allocs, 82,267 frees, 4,200,256,957 bytes allocated
> ==25905== 
> ==25905== 41 (32 direct, 9 indirect) bytes in 1 blocks are definitely lost in loss record 696 of 775
> ==25905==    at 0x4C2AB9D: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==25905==    by 0x4ECF846: guestfs_int_safe_realloc (alloc.c:80)
> ==25905==    by 0x4ED05FF: find_or_cache_feature (available.c:48)
> ==25905==    by 0x4ED073C: guestfs_impl_feature_available (available.c:89)
> ==25905==    by 0x4EA4B21: guestfs_feature_available (actions-5.c:1131)
> ==25905==    by 0x4EF9FDF: guestfs_impl_list_filesystems (listfs.c:49)
> ==25905==    by 0x4E7EDF2: guestfs_list_filesystems (actions-3.c:419)
> ==25905==    by 0x4EE0FD2: guestfs_impl_inspect_os (inspect.c:62)
> ==25905==    by 0x4E50E12: guestfs_inspect_os (actions-0.c:500)
> ==25905==    by 0x4E3292: ocaml_guestfs_inspect_os (guestfs-c-actions.c:9385)
> ==25905==    by 0x4696C8: camlGuestfs__fun_16039 (guestfs.ml:1242)
> ==25905==    by 0x423224: camlV2v__inspect_source_1085 (v2v.ml:306)
> 
> I've been staring at the code for a long while, and I cannot see what
> the problem is ...

It's officially driving me nuts.  I'm beginning to wonder if it's
memory corruption, but perhaps it's just because it's late at night.
The reproducer is simple enough:

$ make -C test-data check
$ ./run valgrind --log-file=/tmp/log \
      --suppressions=valgrind-suppressions \
      --leak-check=full \
      virt-v2v -ic test:///$(pwd)/test-data/phony-guests/guests.xml windows \
      -o null
$ less /tmp/log

(/tmp/log is attached)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
-------------- next part --------------
==30959== Memcheck, a memory error detector
==30959== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30959== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==30959== Command: virt-v2v -ic test:///home/rjones/d/libguestfs/test-data/phony-guests/guests.xml windows -o null
==30959== Parent PID: 26912
==30959== 
==31022== 
==31022== HEAP SUMMARY:
==31022==     in use at exit: 6,768,875 bytes in 1,906 blocks
==31022==   total heap usage: 70,532 allocs, 68,626 frees, 14,316,248 bytes allocated
==31022== 
==31022== LEAK SUMMARY:
==31022==    definitely lost: 0 bytes in 0 blocks
==31022==    indirectly lost: 0 bytes in 0 blocks
==31022==      possibly lost: 0 bytes in 0 blocks
==31022==    still reachable: 2,719,189 bytes in 1,019 blocks
==31022==         suppressed: 4,049,686 bytes in 887 blocks
==31022== Reachable blocks (those to which a pointer was found) are not shown.
==31022== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==31022== 
==31022== For counts of detected and suppressed errors, rerun with: -v
==31022== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 19 from 19)
==30959== 
==30959== HEAP SUMMARY:
==30959==     in use at exit: 7,996,556 bytes in 1,679 blocks
==30959==   total heap usage: 77,300 allocs, 75,621 frees, 1,666,590,610 bytes allocated
==30959== 
==30959== 41 (32 direct, 9 indirect) bytes in 1 blocks are definitely lost in loss record 696 of 780
==30959==    at 0x4C2AB9D: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30959==    by 0x4ECF846: guestfs_int_safe_realloc (alloc.c:80)
==30959==    by 0x4ED05FF: find_or_cache_feature (available.c:49)
==30959==    by 0x4ED073C: guestfs_impl_feature_available (available.c:90)
==30959==    by 0x4EA4B21: guestfs_feature_available (actions-5.c:1131)
==30959==    by 0x4EF9FDF: guestfs_impl_list_filesystems (listfs.c:49)
==30959==    by 0x4E7EDF2: guestfs_list_filesystems (actions-3.c:419)
==30959==    by 0x4EE0FD2: guestfs_impl_inspect_os (inspect.c:62)
==30959==    by 0x4E50E12: guestfs_inspect_os (actions-0.c:500)
==30959==    by 0x4E3292: ocaml_guestfs_inspect_os (guestfs-c-actions.c:9385)
==30959==    by 0x4696C8: camlGuestfs__fun_16039 (guestfs.ml:1242)
==30959==    by 0x423224: camlV2v__inspect_source_1085 (v2v.ml:306)
==30959== 
==30959== LEAK SUMMARY:
==30959==    definitely lost: 32 bytes in 1 blocks
==30959==    indirectly lost: 9 bytes in 2 blocks
==30959==      possibly lost: 0 bytes in 0 blocks
==30959==    still reachable: 3,954,028 bytes in 882 blocks
==30959==         suppressed: 4,042,487 bytes in 794 blocks
==30959== Reachable blocks (those to which a pointer was found) are not shown.
==30959== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==30959== 
==30959== For counts of detected and suppressed errors, rerun with: -v
==30959== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 1 from 1)