[libvirt] [PATCH 2/3] travis: Use pre-built Docker image

[Daniel P. Berrangé]

On Wed, Jun 13, 2018 at 02:58:55PM +0200, Andrea Bolognani wrote:
> On Wed, 2018-06-13 at 11:32 +0100, Daniel P. Berrangé wrote:
> > On Tue, Jun 12, 2018 at 12:12:12PM +0200, Andrea Bolognani wrote:
> > > The pre-built images have been hand-crafted using the
> > > build dependencies recorded in the libvirt-jenkins-ci
> > > repository: of course that's not something that we want
> > > to keep doing manually going forward, so figuring out a
> > > sensible way to generate Dockerfiles and potentially
> > > even Docker images automatically is pretty high on the
> > > priority list.
> > 
> > When first testing I produced a custom Ubuntu docker image
> > with not much effort. I was just creating  a file in
> > "libvirt-jenkins-ci" repo called "images/ubuntu-18.04.docker"
> > that contains
> > 
> >   FROM ubuntu:18.04
> >   RUN apt-get update
> >   ENV PACKAGES \
> >      ::PACKAGE-LIST:: \
> >   RUN apt-get -y install $PACKAGES
> 
> Pretty much exactly how I've created the images you can find on
> Docker Hub, except for
> 
> >   RUN mkdir /build
> >   WORKDIR /build
> 
> this bit, which AFAICT is entirely unnecessary.

WORKDIR /build, avoids the need for the '-w /build' arg in
the .travis.yml docker run command. I think there's a slight
plus to having the workdir set automatically, avoiding the
need for a -w arg.

> > ::PACKAGE-LIST:: can be built by reading the
> > guests/vars/projects/libvirt.yml file, and then
> > expanding it based on guest/vars/mappings.yml
> > 
> > I hadn't written code for that bit, but it just
> > needs a short python script to read the two
> > yaml files and map the data sets.
> 
> Yeah, same here: I just extracted the package list from the output
> of a 'lcitool update' run this time around, but I was planning on
> writing a tool to do that for me just like you mention.
> 
> The one thing I haven't quite figured out yet is where to store
> the resulting Dockerfiles. If we committed them to some repository
> we could take advantage of Docker Hub's autobuild support, which
> would be pretty neat; on the other hand, being generated content,
> they have no business being committed, plus it would be tricky to
> ensure the generated files are always in sync with the source
> mappings without introducing a bunch of scaffoling to the
> libvirt-jenkins-ci repository.

I think we should just have the dockerfile templates (ie with
the ::PACKAGE:: placeholder) in the libvirt-jenkins-ci repo.
We don't need to store the expanded dockerfile. Then we can have
a CI job somewhere that automatically rebuilds the & uploads new
docker images whenever a change is pushed to libvirt-jenkins-ci.

> > I was only going to do packages forthe libvirt.yml,
> > but we can expand to cover the other modules too
> > quite easily, as its just taking the union of all
> > the project files.
> 
> I don't think we can/want to do that.
> 
> The way build dependencies for projects are set up right now, we
> expect to build eg. libvirt-glib against our own copy of libvirt
> rather than the distro-provided one, so libvirt is *not* included
> among the packages required by the libvirt-glib project.
> 
> So if we included build dependencies for all projects in the Docker
> images, that would make them way bigger and you would still be
> unable to build libvirt-glib or whatever on top of them. Perhaps we
> can find some way out of this later, but I'd rather move one step
> at the time instead of trying to solve all the things in one fell
> swoop :)

Yeah, lets only focus on core libvirt right now until we have an
immediate need for more.

> > ie rather than
> > 
> >  Name: libvirt/build
> >  Tag: ubuntu-18.04
> > 
> > we should have
> > 
> >   Name: libvirt/ubuntu
> >   Tag: 18.04
> 
> I don't mind having several images and using the tag only for the
> version number, if that's something that will make the result look
> less alien to Docker users; however, I think we should keep the
> names consistent with what we use on our CentOS CI, so it would be
> ubuntu:18 instead of ubuntu:18.04.

NB that is ambiguous as Ubuntu does two releases a year, 18.04 and
18.10

> > Though perhaps make clear it is for CI, so
> > 
> >   Name: libvirt/ci-ubuntu
> >   Tag: 18.04
> 
> Just like for the guests you can create and manage with lcitool,
> while these images will be primarily used for Travis CI they
> should be usable for developers as well, which is why I picked the
> name "build" instead of "ci" or "travis-ci" in the first place.

Sure, 'libvirt/build-'  is fine

> At the same time, I didn't use just the distribution name because
> I didn't want to give the impression that by pulling them you
> would get an OS with libvirt already installed.

Agreed.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|