[libvirt PATCH 2/3] cirrus: Add templates and refresh script

Erik Skultety eskultet at redhat.com
Mon Jun 29 16:08:27 UTC 2020


On Mon, Jun 29, 2020 at 03:32:22PM +0100, Daniel P. Berrangé wrote:
> On Mon, Jun 29, 2020 at 03:58:43PM +0200, Andrea Bolognani wrote:
> > This is similar to what we already use for Dockerfiles, with one
> > key difference: while we still rely on lcitool taking care of the
> > complicated work for us, in this case we're only provided with a
> > bunch of variables and we have to do the last bit of work (that
> > is replacing them inside an existing template) ourselves.
> >
> > Signed-off-by: Andrea Bolognani <abologna at redhat.com>
> > ---
> >  ci/cirrus/refresh                  | 32 ++++++++++++++++++++++++++++++
> >  ci/cirrus/templates/freebsd-12.yml | 25 +++++++++++++++++++++++
> >  ci/cirrus/templates/macos-1015.yml | 27 +++++++++++++++++++++++++
>
> These files are largely the identical content to the same name
> files in the dir above which feels like it is easy to optimize
> away.
>
> >  3 files changed, 84 insertions(+)
> >  create mode 100755 ci/cirrus/refresh
> >  create mode 100644 ci/cirrus/templates/freebsd-12.yml
> >  create mode 100644 ci/cirrus/templates/macos-1015.yml
> >
> > diff --git a/ci/cirrus/refresh b/ci/cirrus/refresh
> > new file mode 100755
> > index 0000000000..51deca94ac
> > --- /dev/null
> > +++ b/ci/cirrus/refresh
> > @@ -0,0 +1,32 @@
> > +#!/bin/sh
> > +
> > +if test -z "$1"
> > +then
> > +    echo "syntax: $0 PATH-TO-LCITOOL"
> > +    exit 1
> > +fi
> > +
> > +LCITOOL=$1
> > +
> > +if ! test -x "$LCITOOL"
> > +then
> > +    echo "$LCITOOL is not executable"
> > +    exit 1
> > +fi
> > +
> > +for infile in templates/*
> > +do
> > +    outfile="${infile##*/}.j2"
> > +    host="${outfile%%.*}"
> > +
> > +    eval $("$LCITOOL" dockerfile "libvirt-$host" libvirt --variables)
> > +
> > +    sed -e "s|[@]PKGS@|$PKGS|g" \
> > +        -e "s|[@]CROSS_PKGS@|$CROSS_PKGS|g" \
> > +        -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g" \
> > +        -e "s|[@]CPAN_PKGS@|$CPAN_PKGS|g" \
> > +        -e "s|[@]MAKE@|$MAKE|g" \
> > +        -e "s|[@]NINJA@|$NINJA|g" \
> > +        -e "s|[@]PYTHON@|$PYTHON|g" \
> > +        <"$infile" >"$outfile"
> > +done
>
> I feel like this should really be reduced to just:
>
>   $LCITOOL cirrusci libvirt-$host libvirt > $outfile
>
> where the 'cirrusci' command generates this content....

I agree with the idea of having a new command generating the content below, I
tend to look at this as different generator plugins/backends that we support,
so having a dedicated command makes much more sense to me.

>
> > diff --git a/ci/cirrus/templates/freebsd-12.yml b/ci/cirrus/templates/freebsd-12.yml
> > new file mode 100644
> > index 0000000000..228ea67827
> > --- /dev/null
> > +++ b/ci/cirrus/templates/freebsd-12.yml
> > @@ -0,0 +1,25 @@
> > +freebsd_instance:
> > +  image_family: freebsd-12-1
> > +
> > +env:
> > +  CI_REPOSITORY_URL: {{ CI_REPOSITORY_URL }}
> > +  CI_COMMIT_REF_NAME: {{ CI_COMMIT_REF_NAME }}
> > +  CI_COMMIT_SHA: {{ CI_COMMIT_SHA }}
> > +  PKGS: @PKGS@
> > +  MAKE: @MAKE@
> > +  PYTHON: @PYTHON@
> > +
> > +freebsd_12_task:
> > +  install_script:
> > +    - pkg install -y $PKGS
> > +  clone_script:
> > +    - git clone --depth 100 "$CI_REPOSITORY_URL" .
> > +    - git fetch origin "$CI_COMMIT_REF_NAME"
> > +    - git reset --hard "$CI_COMMIT_SHA"
>
> ...down to this point.
>
>
> > +  build_script:
> > +    - mkdir build
> > +    - cd build
> > +    - ../autogen.sh --prefix=$(pwd)/install-root
> > +    - $MAKE -j3
> > +    - $MAKE -j3 install
> > +    - $MAKE -j3 dist
>
> This part can then be stored in ci/cirrus/build.yml since it is
> common to freebsd & macos.
>
> So now in gitlab-ci.yml we can just concatenate the two into a
> temp file:
>
>    cat ci/cirrus/$NAME.yml ci/cirrus/build.yml >  ci/cirrus/$NAME.yml.j2
>    cirrus-run ci/cirrus/$NAME.yml.j2

How about instead of storing the same build.yml in every secondary project's ci
directory we store the build.yml in libvirt-ci repo and generate the whole
thing with lcitool? I think the way the build YAML would look like for every
single subproject, we could achieve the level of similarity we have for lcitool
native builds - IOW the build job abstraction we have across projects under
guests/playbooks/build/projects could also be achieved here with cirrus, just
an idea.

Erik




More information about the libvir-list mailing list