Test failures on macOS 12

[Christophe de Dinechin]

> On 10 Jun 2022, at 11:34, Andrea Bolognani <abologna at redhat.com> wrote:
> 
> On Fri, May 06, 2022 at 03:00:14AM -0700, Andrea Bolognani wrote:
>> I'm trying to enable CI coverage for macOS 12, but I'm running into a
>> couple of issues that I'm not sure how to handle.
>> 
>> Note that the test suite currently passes on macOS 11[1], so these
>> failures have to be a consequence to changes made to macOS that we
>> haven't yet learned how to cope with.
>> 
>> The first one is in vircryptotest:
>> 
>>  Encrypt aes265cbc ... Expected ciphertext doesn't match
>> 
>> I've added some debug statements and it looks like the generated data
>> is different every time, which seems like a pretty good indication
>> that virrandommock is not being picked up correctly. This is not the
>> only test program that uses that specific mock though, so I'm not
>> sure what makes it fail when all the others are succeeding.
>> 
>> The other issue is in qemuxml2argvtest:
>> 
>>  error : virCommandWait:2752 : internal error: Child process
>>    (/usr/libexec/qemu/vhost-user/test-vhost-user-gpu --print-capabilities)
>>    unexpected fatal signal 6: dyld[8896]: symbol not found in flat
>>    namespace '_virQEMUCapsGet'
>>  error : qemuVhostUserFillDomainGPU:394 : operation failed: Unable to
>>    find a satisfying vhost-user-gpu

I don’t see this one.

What I see instead is failures on qemucapabilitiestest.
The tests seems to depend on a particular install path for the
qemu-system- binaries:

    binary = g_strdup_printf("/usr/bin/qemu-system-%s",
                             data->archName);

On macOS, there is something called “system integrity protection”
which makes it a bit more difficult to add stuff to /usr/bin.

However, I don’t really think that’s the problem. I ran some tests with:

VIR_TEST_DEBUG=1 VIR_TEST_RANGE=1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69 /Users/ddd/Work/libvirt/build/tests/qemucapabilitiestest

The error I get seem to point to ‘hvf’ been unexected in the output:

15) 6.2.0 (x86_64)                                                    ... 
In '/Users/ddd/Work/libvirt/tests/qemucapabilitiesdata/caps_6.2.0.x86_64.xml':
Offset 7557
Expect [c]
Actual [hvf'/>
  <flag name=‘c]

I do not really understand what the test is supposed to do. It seems
to be comparing with reference files, but where does it get the
qemu capabilities to compare with from?

How should I proceed?

>> 
>> So the various virFileWrapperAddPrefix() calls that cause the
>> contents of tests/qemuvhostuserdata/ to override the host's own
>> vhostuser configuration are still effective, but for some reason the
>> trivial test-vhost-user-gpu shell script can't be run successfully
>> because an internal libvirt symbol can't be found somehow? Confusing.
>> 
>> Roman, does any of this ring a bell? Any chance you could
>> investigate? macOS 12 has been out for a while now so I'd be very
>> keen to have it added to CI.
> 
> Roman, any chance you can find some time to investigate the test
> failures documented above? I'm afraid I've reached the limit of my
> ability to debug macOS-specific behavior :(
> 
> -- 
> Andrea Bolognani / Red Hat / Virtualization
>