[libvirt PATCH v2 6/9] gitlab: add x86_64 native CI jobs

[Andrea Bolognani]

On Wed, 2020-03-25 at 10:44 +0000, Daniel P. Berrangé wrote:
> On Wed, Mar 25, 2020 at 11:27:31AM +0100, Andrea Bolognani wrote:
> > On Wed, 2020-03-25 at 09:35 +0100, Erik Skultety wrote:
> > > Related to the availability of shared runners, for merge_requests/post_merge
> > > builds only, the PSI/OpenShift infra may actually help in order to achieve
> > > more stable execution times due to the usage of private runners.
> > 
> > As explained by Dan, that will not help for the merge request builds,
> > because those are executed in the context of the submitter's fork and
> > as such don't have access to any runners we might deploy ourselves.
> > 
> > It would help speed up the post merge phase, but once we move to a
> > merge request workflow we'll want to test as much as possible at the
> > time the merge request is submitted, and there won't be much that
> > needs to happen after the changes hit master... So dedicated runners
> > might actually not be useful outside of supporting non-Linux targets.
> 
> NB, our current testing is all just build+unit testing. We want to expand
> to do integration testing, and that will also benefit from dedicated runners
> so that we can actually run stuff in a real OS, as opposed to a container
> where our functionality is much reduced.

Good point, I was just thinking in terms of migrating what we already
have to GitLab rather than building upon it.

This kind of testing will still be limited by the fact that merge
requests will not be able to use these dedicated runners. Compare
this to something like KubeVirt's CI, where proper integrations tests
are executed for each and every merge request...

> Also, even with merge request testing, we'll still want to test everything
> post-merge, as there is still a gap there.  IIUC, the merge requests will
> test against the git master than that the merge request branch was based
> on. The fast-forward to current git master only happens after that.
> 
> So there is a risk that merge request can pass testing but still result
> in breakage due to change to git master affecting the fast forward result.
> 
> There might be ways to mitigate this by having a CI job itself try to do
> a fast-forwar to latest git master, rather than honouring the git state
> given to it, but I've not investigated.

There has to be a way to tell GitLab to re-test the actual code that
is about to hit master when a merge request is approved... It would
be kind of insane otherwise, since both the base branch and the set
of CI tests themselves might be weeks, if not months old by the time
a merge request is approved.

-- 
Andrea Bolognani / Red Hat / Virtualization