[PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts

[Erik Skultety]

On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote:
> On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote:
> > After addition of the new libvirt-client-qemu sub-package which is using
> > python bindings (thus creating a circular dependency between the libvirt
> > and libvirt-python projects) the integration jobs fail with:
> > 
> >   Error:
> >    Problem: conflicting requests
> >     - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
> 
> Hmm, I wonder, should we really be trying to install *all* RPMs ?
> 
> IIUC, we're only running integration tests for QEMU, so it ought
> to be sufficient to do only have the QEMU driver and virsh client
> and Perl APIs. eg can we change ci/integrate-templates.yml to
> do
> 
>    createrepo libvirt-rpms
>    createrepo libvirt-perl
>    dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt

Sorry for the delay, I was actually integrating ^this suggestion and GitLab is,
well, slow to test these changes with. Anyhow, I vote for Peter's original
patch as it's the simplest and cleanest way to fix the problem, why? Here's
what I ended up with on my branch:

diff --git a/ci/integration-template.yml b/ci/integration-template.yml
index d4799b2ca8..9c58a1b71c 100644
--- a/ci/integration-template.yml
+++ b/ci/integration-template.yml
@@ -25,7 +25,31 @@


 .install-deps: &install-deps
-  - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
+  - sudo dnf install -y createrepo
+  - createrepo libvirt-rpms
+  - createrepo libvirt-perl-rpms
+  - |-
+    for repo in libvirt-rpms libvirt-perl-rpms; do
+    cat << EOF >> "$SCRATCH_DIR/ci.repo"
+    [CI-$repo]
+    name=ci-$repo
+    baseurl=file://$(pwd)/$repo
+    enabled=1
+    gpgcheck=0
+    EOF
+    done
+  - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/
+  - sudo dnf install -y
+          libvirt-daemon-config-nwfilter
+          libvirt-daemon-config-network
+          libvirt-daemon-driver-interface
+          libvirt-daemon-driver-network
+          libvirt-daemon-driver-nodedev
+          libvirt-daemon-driver-nwfilter
+          libvirt-daemon-driver-qemu
+          libvirt-daemon-driver-secret
+          libvirt-daemon-driver-storage
+          perl-Sys-Virt
   - sudo pip3 install --prefix=/usr avocado-framework


Not only does it require more steps, is ugly because of the heredoc combined
with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu
also doesn't pull all the needed bits as it was suggested, e.g.
libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests
don't need virtinterfaced, I still had to include it because the logic which
restarts all the daemons would have to be adjusted as well; and then one day we
add a test for virtinterfaced we'd have to add it back, so IMO installing and
restarting all the daemons is the cleanest and most transparent way of setting
up the CI env even though we're not making use of all of it now (and so I ended
up naming all the secondary drivers explicitly).

The other huge inconvenience with the proposed approach compared to Peter's is
debugging the failures. What I currently do with any integration CI failures is
that I connect to the runner machine and spin up an instance of one of the
images from which gitlab provisions the environment, download the RPM artifacts
from libvirt and libvirt-perl, install and then debug - again, why? Because
unfortunately trying to do this locally is still less convenient even though
we've done plenty of work in lcitool to improve the status quo. So, back to
your suggestion - I'd have to create the repositories manually, install a few
explicit packages and then test; sorry, that maybe fine for the CI, but doesn't
beat the following in any way:

wget <libvirt-rpms-uri> -O libvirt_rpms.zip
wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip
wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip
unzip <yada yada>

dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*


Regards,
Erik