[libvirt PATCH 00/20] ci: Move GitLab build recipes to a standalone script

[Erik Skultety]

...

> > So, how about I get rid of the multiplexing and CLI parsing by placing each job
> > recipe in a standalone script or even going one step further and extract the
> > commonalities to a separate file that would be source from within each job
> > script? Would you consider that meeting each other halfway?
> 
> Lets consider the core CI (build + unit tests in .gitlab-ci.yml), separately
> from the integration test CI (ci/integration-template.yml).
> 
> For the core CI, I'm just not convinced of the benefit of moving the commands
> out into a shell script, as the set of commands is small and straightforward.
> 
> For the ingration CI though, I can see benefit because of all the command
> logic related to fetching and building dependancies and setup tasks.
> 
> In retrospect this is a sign that we need to introduce a higher level frontend
> for invoking the libvirt-tck tests. We already have avocado as a frontend to
> replace the existing  'libvirt-tck' command line tool, but we never got rid of
> the latter. We should probably get rid of the existing 'libvirt-tck' cli and
> introduce a brand new pure python 'libvirt-tck' cli, with commands for running
> avocado, optionally installing deps, and all the other interesting things devs
> want help with.  The ci/integration-template.yml could then invoke this newly
> re-invented 'libvirt-tck' command and eliminate most of the complexity in the
> integration-template.yml file.

Well, that was the plan ever since June 2021 when the concept of lavocado was
proposed to the list, but it wasn't widely accepted as it appears we haven't
found the common ground back then. IIRC the idea of making lavocado the
new TCK re-incarnation was actually accepted, but the effort on its own didn't
get traction, particularly because IMO there's not that much value spending
time re-writing existing Perl tests to Python without investing time into
improving the environment preparation and hence eliminating the biggest
problem all these frameworks have which is that to some extent they're still
touching the system libvirt on the host, potentially leaving it in an
inconsitent state. And so I've been steering my attention towards making the
necessary changes to lcitool and hopefully make it the virt-stack's universal
environment preparation tool.
As for what the new TCK cli should look like, that's a good question, FWIW I
don't agree we should introduce any logic related to dependency installation
into the environment, nor anything that is environment related - we've done
that with Avocado-VT and we know how it ended; the pure Avocado framework also
provides some of these features which I don't necessarily agree with (Ansible
integration, container execution integration, etc.), but luckily these are not
mandatory. I am a strong advocate of dividing the CI problem into environment
preparation and test execution and not actually trying to combine these two,
it's not going to end up well IMO (we have a precedent) and instead the new
framework needs to do a single thing which is execute the tests and do it well
and do it agnostic to the environment AND to the language in which the tests
have been written - all of which Avocado is very good at.

Now, while I'm very happy that the idea of reworking TCK has come to the table
again, like you mention below, it's not an easy and quick task, so we need some
temporary bandaids before we get there and I'm totally fine dropping much of
the conversion efforts in the future, but I still think it can help us short
term.

> 
> Essentially we need to look at the gitlab CI yml files as being a very thin
> glue layer, and anything interesting should be in the end user facing tools
> developers already use.
> 
> Introducing a new libvirt-tck python cli is likely not a quick fix though.
> So if we split the ci/integration-template.yml commands out into a set of
> bash scripts, that'll give us a good illustration of what commands we would
> want in a new libvirt-tck cli impl. Then as the new libvirt-tck cli arrives,
> we can eliminate the bash scripts from libvirt.git

Yes, agreed.

> 
> I guess this would mean dropping much of this particular series, but keeping
> much of the corresponding integration test series.

How much of this series should be dropped in your opinion, given my vision I
tried to briefly outline above? I admit I got confused a bit by what your
actual expectation from this series is.

Erik

> 
> > As for POSIX compliance, I guess this would be a soft-requirement based on
> > whether shellcheck was run during review, does gitlab do something better? IIUC
> > there's no pre-check and you'll only know after a job was executed that there
> > was an incompatible statement.
> 
> I think its ok if we use bash scripts as a short term solution, with a
> plan to move to a more supportable python impl in future.
> 
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>