[libvirt-glib PATCH] gitlab: introduce CI jobs testing git master & distro libvirt

Andrea Bolognani abologna at redhat.com
Wed Jun 17 14:20:05 UTC 2020


On Wed, 2020-06-17 at 14:35 +0100, Daniel P. Berrangé wrote:
> On Wed, Jun 17, 2020 at 12:54:56PM +0200, Andrea Bolognani wrote:
> > On Thu, 2020-06-11 at 17:42 +0100, Daniel P. Berrangé wrote:
> > > +.script_variables: &script_variables |
> > > +  export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"
> > > +  export VROOT="$SCRATCH_DIR/vroot"
> > > +  export CCACHE_BASEDIR="$(pwd)"
> > > +  export CCACHE_DIR="$CCACHE_BASEDIR/ccache"
> > > +  export CCACHE_MAXSIZE="500M"
> > > +  export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH"
> > > +  export SCRATCH_DIR="/tmp/scratch"
> > > +  export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig"
> > 
> > You need to define $SCRATCH_DIR before $VROOT, otherwise the latter
> > will get the wrong value.
> > 
> > I also just realized that the way we set $CCACHE_BASEDIR might not
> > work for the libvirt build that we perform as a prerequisite, and
> 
> There's no problem - $(pwd) expands at the time the variable is
> defined.

The problem with $CCACHE_BASEDIR being set to $PWD is that the build
that we do for libvirt happens *outside* of that directory, which I
think messes up with the path rewriting and might result in libvirt
builds not taking proper advantage of the cache.

It's all a bit confusing, to be honest, so I could have gotten it
wrong, but IIUC the idea is that if you have multiple built trees
for the same project you'd be able to do something like

  $ cd ~/build/tree/one
  $ export CCACHE_BASEDIR=$PWD
  $ make
  $ cd ~/build/tree/two
  $ export CCACHE_BASEDIR=$PWD
  $ make

and, thanks to $CCACHE_BASEDIR, ccache will be able to figure out
that some of the files are perfect matches and will thus achieve
better hit rates.

This is not our case for a few reasons:

  * we only build each project once per container;

  * we set $CCACHE_BASEDIR once instead of once per build directory;

  * we build libvirt outside of $CCACHE_BASEDIR.

So basically our current use of $CCACHE_BASEDIR is likely useless if
not even somewhat harmful, and we should just get rid of it.

> > since we use the same paths across builds anyway there doesn't seem
> > to be a point in setting it. So I suggest we have
> > 
> >   export SCRATCH_DIR="/tmp/scratch"
> >   export VROOT="$SCRATCH_DIR/vroot"
> >   export CCACHE_DIR="$SCRATCH_DIR/ccache"
> 
> The ccache dir has to be a subdir of the source checkout for
> gitlab to cache it between jobs.

Right! I think you told me about this already, but then I forgot
about it O:-)

Keeping that in mind, we should have

  export CCACHE_DIR="$PWD/ccache"

then.

> > > +  script:
> > > +    - mkdir build
> > > +    - cd build
> > > +    - ../autogen.sh --prefix="$VROOT"
> > > +    - $MAKE install
> > > +    - $MAKE dist
> > 
> > Do we want distcheck here, or is the combination of dist plus
> > building the RPM package (which effectively runs check inside the
> > generated tarball) good enough?
> 
> I think its good enough. Also I want to switch to meson for glib
> asap.

Okay.

Be aware of the fact that, with the switch to Meson, we're going to
need something like

  https://gitlab.com/libosinfo/libosinfo/-/blob/master/.gitlab-ci.yml#L6-13

because Meson, unlike autotools, is properly multiarch-aware.

In fact, a better solution would probably be to bake the multilib
string into the Dockerfile directly, similarly to what we already do
for other multiarch-related information, instead of figuring it out
dynamically at build time. I haven't had the time to look into
implementing that yet, unfortunatley, but maybe we can get that in
before libvirt-glib moves off autotools.

-- 
Andrea Bolognani / Red Hat / Virtualization




More information about the libvir-list mailing list