[libvirt] encountered failed test cases with latest checkout of libvirt

[Michal Privoznik]

On 03/29/2017 09:01 AM, Dan wrote:
> On Wed, Mar 29, 2017 at 1:01 AM, D L <srwx4096 at gmail.com> wrote:
>
>> Hi all,
>>
>> This email content might be duplicated with other thread posted in other
>> places that I do not know yet where to find or search (please let me know
>> if it is true). I am working on a small bug of virsh domxml-to-native.
>> Before I made any changes to the code in order to fix the bug, I 'make
>>  test'-ed and 'valgrind test'-ed the code, encountered four failures in
>> 'make test' and 19 failures in 'make -C tests valgrind'. What I did today
>> was the following:
>>
>> git checkout master
>> git pull
>> ./autogen.sh --prefix=$(pwd)/build
>> make
>> make install
>> # So far so good, no errors ; however
>>
>> make check VIR_TEST_EXPENSIVE=1
>> # generated the following test-suite.log
>> =========================================
>>    libvirt 3.2.0: tests/test-suite.log
>> =========================================
>>
>> # TOTAL: 115
>> # PASS:  111
>> # SKIP:  0
>> # XFAIL: 0
>> # FAIL:  4
>> # XPASS: 0
>> # ERROR: 0
>>
>> .. contents:: :depth: 2
>>
>> FAIL: virfirewalltest
>> =====================
>>
>> TEST: virfirewalltest
>>       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 40  FAIL
>> FAIL virfirewalltest (exit status: 1)
>>
>> FAIL: networkxml2firewalltest
>> =============================
>>
>> TEST: networkxml2firewalltest
>>                                          0   FAIL
>> FAIL networkxml2firewalltest (exit status: 1)
>>
>> FAIL: nwfilterebiptablestest
>> ============================
>>
>> TEST: nwfilterebiptablestest
>>                                          0   FAIL
>> FAIL nwfilterebiptablestest (exit status: 1)
>>
>> FAIL: nwfilterxml2firewalltest
>> ==============================
>>
>> TEST: nwfilterxml2firewalltest
>>                                          0   FAIL
>> FAIL nwfilterxml2firewalltest (exit status: 1)
>>
>>
>>
>> make syntax-check VIR_TEST_EXTENSIVE=1
>> # I did not paste the result here and I do not understand what it means.
>> # available upon request
>>
>> Finally,
>> make -C tests valgrind
>> # generated file was attached.
>>
>> It seems most of the valgrind failures were false positives, I can only
>> tell
>> by comparing the log file with the examples in the HACKING page. But I
>> do not have the knowledge to determine.
>>
>> Could anyone tell me if I am missing something? Having those test failures
>> is a problem or not? Which ones can be safely ignored? Is there any easier
>> way to avoid the false positives? I am concerned because I also need to
>> run the test when I make changes to the code. On the other hand, I would
>> also like to learn the right way to do the testing or whatever it takes to
>> make
>> things right.
>>
>> I am running linux using vmware Workstation Pro 12 on Windows 10.
>> Linux version 4.4.55-1-MANJARO (builduser at manjaro) (gcc version 6.3.1
>> 20170306 (GCC) ) #1 SMP PREEMPT
>>
>>
>>
>> Dan
>>
>>
>>
>> On ubuntu16 and ubuntu14, ebtables were pre-installed. But on
> Archlinux and Debian, ebtables have to be installed explicitly in order to
> pass
> all test cases in 'make check'.  I found out this after I enabled debugging
> checking
> $ make check VIR_TEST_DEBUG=2 VIR_TEST_EXPENSIVE=1

Ah, very well you. I'm not sure what to do here. I mean, we should 
definitely skip networkxml2firewalltest, nwfilterebiptablestest and 
nwfilterxml2firewalltest if EBTABLES is undefined. But the first one is 
slightly more tricky as it can use DBUS when available. Well, we can 
skip it it neither of DBUS and EBTABLES is present. This can be your 
first patch to libvirt ;-)

>
> For the valgrind testing with 'make -C tests valgrind', the errors were
> *mostly* due to
> "FATAL: can't open suppressions file "./.valgrind.supp". I am using
> valgrind 3.12.0.


That's weird, the file should be there. What is a problem though is our 
mocking. Due to some weird glibc-ness, valgrind cannot handle libraries 
that have some unresolved symbols. For instance, if library L that is 
loaded from binary B provides symbols L1 and L2 but rely on symbol B1, 
valgrind (in fact glibc) fails to load the binary even though if run 
without valgrind everything works just fine.

I've looked into this the other day and went from our sources to 
valgrind ones and from there to glibc where I got lost. For instance, 
virpcitest fails because virpcimock uses virAlloc, virFree and other 
internal APIs even though it's not linking with libvirt_util.

Frankly, I've no idea how to fix this. Perhaps Eric have a bright idea?
BTW: linking the mock with libvirt_util fails in exactly the opposite 
case - when not running under valgrind.

Michal