[libvirt] [PATCH 2/2] tests: Document procedure to build QEMU for *.replies generation

Fri Feb 8 17:02:49 UTC 2019

On 2/7/19 3:28 AM, Pavel Hrdina wrote:
> On Wed, Feb 06, 2019 at 03:38:03PM -0500, Cole Robinson wrote:
>> On 1/23/19 3:59 PM, John Ferlan wrote:
>>> Add some comments to describe how to set up the QEMU environment
>>> prior to running the qemucapsprobe in order to allow for the
>>> creation of consistent results.
>>>
>>> Signed-off-by: John Ferlan <jferlan at redhat.com>
>>> ---
>>>  tests/qemucapabilitiestest.c | 34 ++++++++++++++++++++++++++++++++--
>>>  1 file changed, 32 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tests/qemucapabilitiestest.c b/tests/qemucapabilitiestest.c
>>> index 00137bb415..2b6291e99e 100644
>>> --- a/tests/qemucapabilitiestest.c
>>> +++ b/tests/qemucapabilitiestest.c
>>> @@ -200,11 +200,41 @@ mymain(void)
>>>      DO_TEST("riscv64", "caps_3.0.0");
>>>  
>>>      /*
>>> +     * Create a QEMU build environment using a checked out version
>>> +     * of the release tag, such as:
>>> +     *
>>> +     *    git checkout -b v3.0.0 v3.0.0
>>> +     *
>>> +     * Be sure the build dependencies are up to date, such as via yum:
>>> +     *
>>> +     *    yum builddep qemu
>>> +     *
>>> +     * Configure the environment, such as for x86_64:
>>> +     *
>>> +     *   ./configure --target-list=x86_64-softmmu \
>>> +     *      --disable-xen --disable-strip --disable-fdt \
>>> +     *      --disable-werror --enable-debug \
>>> +     *      --enable-system --enable-user --enable-linux-user \
>>> +     *      --with-pkgversion=v3.0.0
>>> +     *
>>
>> AFAICT --enable-system/--enable-user/--enable-linux-user are all
>> redundant if a manual --target-list is specified. I don't know why
>> --disable-fdt matters, nor --enable-debug or --disable-strip. I figure
>> less options the better, defaults should be fine
> 

I dunno - this was just the list I had come up with over time in a local
"remind me how to do that again" file... I also peeked a bit at how Dan
put together those virt-ark packages - it's a bit of digging and
following links from:

https://pagure.io/virt-ark

or

https://copr.fedorainfracloud.org/coprs/berrange/virt-ark/

to find

https://pagure.io/virt-ark/blob/master/f/qemu-ark.spec.in

that it seems the build uses:

./configure --prefix=/opt/qemu/%{version} \
+	    --target-list=x86_64-softmmu \
+	    --disable-xen \
+	    --disable-strip \
+	    --disable-fdt \
+	    --disable-werror

> I agree that this looks weird.  I was trying to figure out how to make
> sure that QEMU is compiled with the same configuration and relying on
> defaults with installing distribution build dependencies is not the
> answer.  Sure, it helps a lot to install most of the dependencies but
> for example, if someone is using CentOS or older version of Fedora
> "yum/dnf builddep qemu" would not result in the same configuration.

Without a bunch of -devel packages installed - the qemu default seems to
be fairly bare. As for yum/dnf - yeah that's obviously specific ;-).

I recall using 'builddep' once and got an error about needing some other
package installed type message, but I don't recall what it was now. But
getting that led me to:

dnf install  $(grep ^BuildRequires libvirt.spec.in| \
             cut -f2 -d' '|grep -v "\/usr/bin" | grep -v "\/usr/sbin")

Unfortunately as time as gone on that isn't sufficient - I have a
feeling checks in the m4 files haven't always translated into the
spec.in file. Recently I had the need to try and enable as much as
possible with a libvirt build and even after all the above I found I
needed to :

dnf install gettext-devel xhtml1-dtds libtool rpcgen gnutls-devel \
            libnl3-devel cppi dnsmasq libxslt perl-XML-XPath

just to get enough to allow ./autogen.sh to complete. Then if I wanted
as much as possible, I had to (sorry for the line break stuff - email
compose is less wide than my saved command):

dnf install libblkid-devel libcap-devel libcurl-devel device-mapper-devel \
            device-mapper-multipath-devel fuse-devel glusterfs-api-devel \
            glusterfs-devel parted-devel python3-devel python2-devel \
            libxml2-devel libssh-devel libpciaccess-devel dbus-devel \
            libssh2-devel numactl-devel sanlock-devel libtirpc-devel \
            yajl-devel libiscsi-devel libcap-ng-devel librados-devel
zfs-fuse \
            libpcap-devel iscsi-initiator-utils librbd-devel
audit-libs-devel \
            polkit-devel

It's an interesting "exercise' to start from ground zero every once in a
while.  I'm sure the CI builds have a list of things that get installed
for a more complete environment, but I didn't peek at that list.

> 
> I was playing with an idea to create some script that would run
> './configure --help' to get all options, the script would have a list
> of options that should be ignored, enabled or disabled and all the other
> options would be automatically enabled.  This would make sure that
> everything is explicitly enabled and ./configure would complain if there
> is any missing dependency and also it would cover all new dependencies.
> 
> The rest of the documentation looks good.
> 
> Pavel
> 
>>> +     * Build the QEMU emulator binary.
>>> +     *
>>> +     * Then from a clean libvirt build:
>>> +     *
>>>       * Run "tests/qemucapsprobe /path/to/qemu/binary >foo.replies"
>>> -     * to generate updated or new *.replies data files.
>>> +     * to generate updated or new *.replies data files, such as:
>>> +     *
>>> +     *    tests/qemucapsprobe /path/to/qemu/binary > \
>>> +     *          tests/qemucapabilitiesdata/caps_3.0.0.x86_64.replies
>>>       *
>>> -     * If you manually edit replies files you can run
>>> +     * If you needed to manually edit replies files you can run
>>>       * "tests/qemucapsfixreplies foo.replies" to fix the replies ids.
>>> +     *
>>> +     * Run "VIR_TEST_REGENERATE_OUTPUT=1 tests/qemucapabilitiestest" to
>>> +     * update the corresponding tests/qemucapabilitiesdata/caps_*.xml file.
>>> +     *
>>> +     * May also need to run "VIR_TEST_REGENERATE_OUTPUT=1 tests/domaincapstest"
>>> +     * depending on what changed.
>>>       */
>>>  
>>
>> If we are getting this detailed, maybe also mention extending
>> qemucaps2xmltest.c and qemucapabilitiestest.c for newly added
>> capabilities files. Up to you

Hmm... true, it's a complex and intertwined pile of files.

Like 1/2 - I'll still hold on to this for a bit more - maybe more
brilliant ideas will be generated ;-)

John

>>
>> Reviewed-by: Cole Robinson <crobinso at redhat.com>
>>
>> - Cole
>>
>> --
>> libvir-list mailing list
>> libvir-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/libvir-list