[libvirt PATCH 00/20] ci: Move GitLab build recipes to a standalone script

[Daniel P. Berrangé]

On Mon, Feb 06, 2023 at 02:52:57PM +0100, Erik Skultety wrote:
> This is a follow up to:
> https://listman.redhat.com/archives/libvir-list/2023-January/237201.html
> 
> The effort here is to unify the way builds/tests are executed in GitLab CI vs
> local container executions and make another step forward in terms of
> reproducibility of (specifically) GitLab environments.
> 
> Even though code to run all but one (coverity) jobs from GitLab via the
> build.sh script is added with this series, local behavior remains the same as
> before this series. The reason for that is that that will require more patches
> ridding of the Makefile which is currently used and instead integrate usage of
> lcitool with the ci/helper Python script which is currently the entry point for
> local container executions.

snip

>  .gitlab-ci.yml |  56 ++++++++++-------------
>  ci/Makefile    |  16 ++++---
>  ci/build.sh    | 121 +++++++++++++++++++++++++++++++++++++++++++------
>  ci/helper      |  21 ++++++---
>  4 files changed, 155 insertions(+), 59 deletions(-)
>  mode change 100644 => 100755 ci/build.sh

I'm really super unenthusiastic about this change, and also the similar
change to add an ci/integration.sh. Shell scripting is something we
worked hard to eliminate from libvirt. It is an awful language
to do anything non-trivial with, error handling, lack of data
structures, variable handling, portability are all bug generators.

I know the .gitlab-ci.yml  'script' commands are technically shell
script, but they are pretty trivial bits and don't have to worry
about portability for dash vs bash etc, or complex control logic.
The majority of it is just a simple list of commands to invoke,
with the occasional conditional.

The build.sh script is by contrast significantly more complex. By
the very nature of taking "N" separate gitlab jobs and multiplexing
them onto the one shell script, we're now having todo command line
arg parsing in shell and de-multiplexing back to commands. The CI
logic is now split up across even more sources - the gitlab config,
the build.sh script and the meson.build files, which I think is
worse for maintaining this too. 

I appreciate the goal of being able to run CI commands locally, but
I'm not feeling this approach is a net win. I'd much rather just
having developers invoke the meson command directly, which is not
that hard.

If we do really want to provide something that 100% mirrors the
gitlab CI job commands though, I'm even more convinced now that
we should be using the .gitlab-ci.yml as the canonical source.

Since I last mentioned that idea, I found out that this should
be something we can achieve via the gitlab runner 'exec' command.

I've not quite figured out the right incantations though. I can
get it to work for a simple repo, but not for the lbivirt.git
yet as it seems to not find the included yml files.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|