[libvirt PATCH 4/4] gitlab-ci: Introduce a new test 'integration' pipeline stage

[Andrea Bolognani]

On Wed, Mar 02, 2022 at 03:52:49PM +0100, Erik Skultety wrote:
> On Wed, Mar 02, 2022 at 06:27:04AM -0800, Andrea Bolognani wrote:
> > I don't think we can expect integration tests to be merged at the
> > same time as a feature when new APIs are involved. If tests are
> > written in Python, then the Python bindings need to introduce support
> > for the new API before the test can exist, and that can't happen
> > until the feature has been merged.
>
> Again, that is a logical conclusion which brings us to an unrelated process
> question: How do we change the contribution process so that the contribution of
> a feature doesn't end with it being merged to the C library, IOW we'd ideally
> want to have a test introduced with every feature,but given that we'd need the
> bindings first to actually do that, but we can't have a binding unless the C
> counterpart is already merged, how do we keep the contributors motivated
> enough? (Note that it's food for thought, it's only tangential to this effort)

Yeah, let's save this massive topic for another time :)

> > If the feature or bug fix doesn't require new APIs to be introduced
> > this is of course not an issue. Most changes should fall into this
> > bucket.
> >
> > So overall I still think using existing artifacts would be the better
> > approach, at least initially. We can always change things later if we
> > find that we've outgrown it.
>
> So, given that https://gitlab.com/libvirt/libvirt-perl/-/merge_requests/55 was
> already merged we should not get to a situation where no artifacts would be
> available because gitlab documents that even if artifacts expired they won't be
> deleted until new artifacts become available, I think we can depend on the
> latest available artifacts without building them. I'll refresh the patch
> accordingly and test.

Thanks!

> > I think it's not unreasonable that when Red Hat, or any other entity,
> > provides hardware access to the project there will be some strings
> > attached. This is already the case for GitLab and Cirrus CI, for
> > example.
> >
> > I could easily see the instance of libvirt-gitlab-executor running on
> > hardware owned and operated by Red Hat returning a failure if a job
> > submitted to it comes with DISTRO=debian-11.
>
> libvirt-gitlab-executor is supposed to be system owner agnostic, I'd even like
> to make the project part of the libvirt gitlab namespace umbrella so that
> anyone can use it to prepare their machine to be plugged into and used for
> libvirt upstream integration testing.

I absolutely agree and it was always my understanding that the
project living under your personal account was only a temporary
measure.

> Therefore, I don't think the project is
> the right place for such checks, those should IMO live solely in the
> gitlab-ci.yml configuration.

To be clear, I didn't mean that the software itself should contain
hardcoded limits on what jobs it will process, but rather that it
would make sense for it to offer a configuration setting along the
lines of

  accept_distros:
    - "alpine-*"
    - "debian-*"

that can be used by the local sysadmin to define such a policy.

I guess this is already sort of already implicitly implemented due to
the fact that a job will fail if the corresponding VM template does
not exist...

-- 
Andrea Bolognani / Red Hat / Virtualization