[libvirt PATCH 4/4] gitlab-ci: Introduce a new test 'integration' pipeline stage

[Jiri Denemark]

On Wed, Mar 02, 2022 at 09:43:24 +0000, Daniel P. Berrangé wrote:
> On Wed, Mar 02, 2022 at 08:42:30AM +0100, Erik Skultety wrote:
> > ...
> Ultimately when we switch to using merge requests, the integration tests
> should be run as a gating job, triggered from the merge train when the
> code gets applied to git, so that we prevent regressions actually making
> it into git master at all.
> 
> Post-merge integration testing always exhibits the problem that people will
> consider it somebody else's problem to fix the regression. So effectively
> whoever creates the integration testing system ends up burdened with the
> job of investigating failures and finding someone to poke to fix it. With
> it run pre-merge then whoever wants to get their code merged needs to
> investigate the problems. Now sometimes the problems with of course be
> with the integration test system itself, not the submitters code, but
> this is OK because it leads to situation where the job of maintaining
> the integration tests are more equitably spread across all involved and
> builds a mindset that functional / integration testing is a critical
> part of delivering code, which is something we've lacked for too long
> in libvirt.

This is all fine as long as there's a way to explicitly merge something
even if CI fails. I don't like waiting for a fix in something completely
unrelated. For example, a MR with translations or an important bug fix
should not be blocked by a bug in CI or an infrastructure issue. So
having a way (restricted, of course) to merge even if pipeline failed
would solve this issue.

Jirka