[libvirt] [PATCH 0/3] libxl: Add a test suite for libxl_domain_config generator

[Jim Fehlig]

On 07/26/2017 05:07 PM, Marek Marczykowski-Górecki wrote:
> On Fri, Jul 14, 2017 at 05:59:57PM -0600, Jim Fehlig wrote:
>> On 07/01/2017 08:16 PM, Marek Marczykowski-Górecki wrote:
>>> On Sun, Feb 26, 2017 at 07:02:24PM -0700, Jim Fehlig wrote:
>>>> Long ago danpb posted some patches to test libvirt domXML to
>>>> libxl_domain_config conversion
>>>>
>>>> https://www.redhat.com/archives/libvir-list/2014-May/msg01102.html
>>>>
>>>> Some of the prerequisite patches were pushed, but we've never managed
>>>> to push patches actually providing the conversion tests. I sent several
>>>> follow-ups to Dan's work but never converged on a satisfactory solution
>>>> for all the Xen versions supported by libvirt. The last attempt was in
>>>> Sept 2014
>>>>
>>>> https://www.redhat.com/archives/libvir-list/2014-September/msg00698.html
>>>>
>>>> I tried to revive the work in Jan 2015, but that also stalled
>>>>
>>>> https://www.redhat.com/archives/libvir-list/2015-January/msg00924.html
>>>>
>>>> Fast-forward over 2.5 years from the first attempt and libvirt no longer
>>>> supports older Xen versions 4.2 and 4.3 that were proving to be problematic.
>>>> Starting with Xen 4.5 libxl added support for libxl_domain_config_from_json,
>>>> which provides a way to implement the conversion tests that work with all
>>>> Xen versions >= 4.5 (including latest xen.git master).
>>>
>>> Few more months have passed...
>>
>> And a few more weeks :-). Sorry for the delay. Slowly catching up on libvirt
>> mail after some time away...
> 
> It's me this time...
> 
>>> FWIW, I've tested it with Xen 4.6. The patch needs very minor update:
>>>    - s/VIRT_TEST_MAIN_PRELOAD/VIR_TEST_MAIN_PRELOAD/
>>>    - add xencaps argument to libxlBuildDomainConfig cal >
>>>
>>> After that, it works! When I made some test to fail, reported error is
>>> not so helpful ("libvirt: Xen Light Driver error : internal error:
>>> Expected and actual libxl_domain_config objects do not compare"), but it
>>> do catch failures.
>>> Then, if I change strcmp to virTestCompareToString, the output is much
>>> more helpful.
>>
>> Thanks for the improvements. I noticed you included them in your V2 of patch
>> 3/3. I assume you needed patches 1 and 2 as well?
> 
> Yes, in this tests shape. But see below.
> 
>>> I'd really love to have it merged, mostly because I want to add more tests
>>> using this framework (see "Add setting CPU features (CPUID) with
>>> libxenlight driver" thread).
>>
>> I really dislike patches 1 and 2. They remove very useful checks IMO.
>> Shortly after posting the series in Feb, I tried mocking the emulator
>> checks. I failed quite a bit before coming up with something that worked,
>> yet not so satisfying :-). I'll attach the (now rather old) patches for
>> reference.
> 
> My independent try to this test approach ("tests: check domain XML to
> libxl structures conversion") had much simpler (but uglier) solution for
> this problem: use /bin/true as emulator path (replaced in runtime, just
> after loading XML from file).

Ah, clever and devious. For that matter all the test files could be changed to 
<emulator>/bin/true</emulator> :-).

>> Do you have time to update these old patches, test, and repost to the list?
>> Or perhaps have better ideas on mocking, or approaches that avoid the need
>> for 1/3 and 2/3.
> 
> If using /bin/true as emulator path is acceptable, I can finish this and
> post updated version.

Did you have any comment on the approach taken in the attachments to this 
thread? Particularly the patch titled "libxl: make emulator checks mockable"?