[libvirt PATCH 4/5] gitlab: add several native CI jobs

[Erik Skultety]

On Fri, Mar 20, 2020 at 03:31:24PM +0000, Daniel P. Berrangé wrote:
> On Fri, Mar 20, 2020 at 04:18:58PM +0100, Erik Skultety wrote:
> > On Fri, Mar 20, 2020 at 02:59:36PM +0000, Daniel P. Berrangé wrote:
> > > On Fri, Mar 20, 2020 at 03:52:15PM +0100, Erik Skultety wrote:
> > > > On Tue, Mar 10, 2020 at 10:09:44AM +0000, Daniel P. Berrangé wrote:
> > > > > With GitLab CI aiming to replace Jenkins and Travis for CI purposes, we
> > > > > need to expand the coverage to include native builds. This patch adds
> > > > > all the jobs currently run in Travis. Compared to Jenkins we obviously
> > > > > miss the FreeBSD jobs, but also Debian 10 and Fedora 30, but we gain the
> > > > > Ubuntu 1804 job as a substitute for Debian.
> > > > >
> > > > > Signed-off-by: Daniel P. Berrangé <berrange at redhat.com>
> > > > > ---
> > > > >  .gitlab-ci.yml | 41 +++++++++++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 41 insertions(+)
> > > > >
> > > > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > > > > index e28ec584ea..3e15d08d17 100644
> > > > > --- a/.gitlab-ci.yml
> > > > > +++ b/.gitlab-ci.yml
> > > > > @@ -1,7 +1,39 @@
> > > > >  stages:
> > > > >    - website
> > > > > +  - native_build
> > > > >    - cross_build
> > > > >
> > > > > +
> > > > > +.native_build_job_template: &native_build_job_definition
> > > > > +  stage: native_build
> > > > > +  script:
> > > > > +    - mkdir build
> > > > > +    - cd build
> > > > > +    - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1)
> > > > > +    - make -j $(getconf _NPROCESSORS_ONLN) syntax-check
> > > > > +    - make -j $(getconf _NPROCESSORS_ONLN) distcheck
> > > >
> > > > I think ^this should more closely follow what we have in the lcitool playbooks,
> > > > e.g. start with:
> > > >     - rm -rf build
> > >
> > > The source tree is already pristine because this is always executed in
> > > a fresh container environment, so there's nothing that will need deleting.
> >
> > Right, but my point was that if e.g. we introduce a FreeBSD builder, we'd want
> > to reference the same job template in which case the directory will exist.
>
> I've not looked in great detail about how gitlab runners work, but something
> on the runner needs to be responsible for checking out the git repo before
> any of this gitlab-ci.yml scrpit runs.
>
> IMHO that can ensure it is "git clean -xdf" so that its state matches what
> we see with the built-in shared runners build environment.

This is achieved by the GIT_CLEAN_FLAGS which, if unspecified, defaults to
-ffdx, so we're covered there.

One more thing that crossed my mind and may speedup the builds a bit - some
time ago I proposed a patch against libvirt-jenkins-ci to enable shallow
cloning which was rejected because of compelling reasons. However, for gitlab
purposes, we can't really do any debugging on the shared runners, so I believe
we may want to apply shallow cloning there, especially since libvirt is quite a
large repo. Gitlab claims [1] the default strategy to be 'fetch' if the
worktree can be found on the disk (this is when used with the 'docker' executor
which should be the case with shared runners). What I couldn't find is how long
are these worktrees cached which could therefore be a factor and we might want
to favour the shallow clones instead.

[1] https://docs.gitlab.com/ee/ci/large_repositories/#git-strategy