[libvirt PATCH v3 08/12] gitlab: add x86_64 native CI jobs

[Daniel P. Berrangé]

On Thu, Mar 26, 2020 at 05:31:48PM +0100, Erik Skultety wrote:
> On Thu, Mar 26, 2020 at 12:35:34PM +0000, Daniel P. Berrangé wrote:
> > This patch adds x86_64 native CI jobs for all distros that we currently
> > build container images for. This is a superset of the Linux jobs run on
> > current Jenkins and Travis platforms.
> >
> > The remaining missing platforms are FreeBSD and macOS, neither of which
> > can use the shared runner container based infrastructure.
> >
> > We may add further native jobs in the future which are not x86_64 based,
> > if we get access to suitable hardware, thus the jobs all have an arch
> > prefix in their name, just like the cross-built jobs do.
> >
> > As with the cross-arch builds, the native jobs are split into two
> > groups. One group is run in all situations, while the other group is
> > only run on the master branch, or branches with a name prefix
> > 'ci-extra-'. This avoids the build time getting too long when
> > developers are testing their code prior to submission, while keeping
> > full coverage of code that is merged.
> >
> > Signed-off-by: Daniel P. Berrangé <berrange at redhat.com>
> > ---
> >  .gitlab-ci.yml | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 70 insertions(+)
> >
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index 631c447793..85ab8424e1 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -4,9 +4,30 @@ variables:
> >
> >  stages:
> >    - prebuild
> > +  - native_build
> >    - cross_build
> >
> >
> > +# Common templates
> > +
> > +# Default native build jobs that are always run
> > +.native_build_default_job_template: &native_build_default_job_definition
> > +  stage: native_build
> > +  script:
> > +    - mkdir build
> > +    - cd build
> > +    - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1)
> > +    - $MAKE -j $(getconf _NPROCESSORS_ONLN) distcheck
> > +
> > +# Extra native build jobs that are only run post-merge, or
> > +# when code is pushed to a branch with "ci-extra-" name prefix
> > +.native_build_extra_job_template: &native_build_extra_job_definition
> > +  <<: *native_build_default_job_definition
> > +  only:
> > +    - master
> > +    - /^ci-extra-.*$/
> 
> As Andrea commented a few patches back, ci-full is probably a better prefix.

Yes, already made this change.


> > +# Native architecture build + test jobs
> > +
> > +x64-debian-9:
> > +  <<: *native_build_extra_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-debian-9:latest
> > +
> > +x64-debian-10:
> > +  <<: *native_build_default_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-debian-10:latest
> > +
> > +x64-debian-sid:
> > +  <<: *native_build_extra_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-debian-sid:latest
> > +
> > +x64-centos-7:
> > +  <<: *native_build_default_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-centos-7:latest
> > +
> > +x64-centos-8:
> > +  <<: *native_build_extra_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-centos-8:latest
> 
> Shouldn't we actually prefer the newer distros over the older ones in terms of
> what runs on all branches vs what runs only on master + a dedicated ci-
> prefixed branch? At least that makes much more sense to from the upstream POV.
> Everything else is just a nice to have.

Counter-intuitively it is actually more important to have the
oldest distros represented, as history has shown that these are
the most likely ones to suffer build breakage. Our developers
are most likely to be doing their own work on the latest versions
of the distros thus detecting problems there.

Thus I've intentionally not picked the latest versions of
everything, and instead tried to get a mixture of vintages,
with a bias to the very oldest and very newest, in the belief
that the intersection of oldest & newest will detect issues
on the middle vintage distros.

CentOS v7 in particular has proved especially important to
test as it is usually our oldest distro.

> > +x64-fedora-30:
> > +  <<: *native_build_default_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-fedora-30:latest
> > +
> > +x64-fedora-31:
> > +  <<: *native_build_extra_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-fedora-31:latest
> 
> Same here...

The combo of Fedora 30 and Fedora rawhide will give good
coverage of F31.

The combo of F31 and rawhide won't be good for F30 


> > +x64-fedora-rawhide:
> > +  <<: *native_build_default_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-fedora-rawhide:latest
> > +
> > +x64-opensuse-151:
> > +  <<: *native_build_default_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-opensuse-151:latest
> > +
> > +x64-ubuntu-1604:
> > +  <<: *native_build_default_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-ubuntu-1604:latest
> > +
> > +x64-ubuntu-1804:
> > +  <<: *native_build_extra_job_definition
> > +  image: quay.io/libvirt/buildenv-libvirt-ubuntu-1804:latest
> 
> ...and here...

I feel Ubuntu 1804 vintage has stronger coverage across our other
default distros, so we'll benefit more from Ubuntu 1604 content
which has less overlap.

> With the distro versions swapped across the job definitions:
> Reviewed-by: Erik Skultety <eskultet at redhat.com>

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|