[libvirt PATCH] ci: Reduce number of stages
Daniel P. Berrangé
berrange at redhat.com
Wed Jun 10 16:46:45 UTC 2020
On Wed, Jun 10, 2020 at 05:15:55PM +0200, Andrea Bolognani wrote:
> On Wed, 2020-06-10 at 13:31 +0100, Daniel P. Berrangé wrote:
> > On Wed, Jun 10, 2020 at 01:14:51PM +0100, Daniel P. Berrangé wrote:
> > > On Wed, Jun 10, 2020 at 01:33:01PM +0200, Andrea Bolognani wrote:
> > > > Building artifacts in a separate pipeline stage also doesn't have any
> > > > advantages, and only delays further stages by a couple of minutes.
> > > > The only job that really makes sense in its own stage is the DCO
> > > > check, because it's extremely fast (less than 1 minute) and, if that
> > > > fails, we can avoid kicking off all other jobs.
> > >
> > > The advantage of using stages is that it makes it easy to see at a
> > > glance where the pipeline was failing.
> Ultimately you'll need to drill down to the actual failure, though,
> so the only situation in which it would really provide value is if
> for some reason *all* cross builds failed at once, which is not
> something that happens frequently enough to optimize for.
> > > > Reducing the number of stages results in significant speedups:
> > > > specifically, going from three stages to two stages reduces the
> > > > overall completion time for a full CI pipeline from ~45 minutes
> > > > to ~30 minutes.
> > > >
> > > >  https://gitlab.com/abologna/libvirt/-/pipelines/154751893
> > > >  https://gitlab.com/abologna/libvirt/-/pipelines/154771173
> > >
> > > I don't think this time comparison is showing a genuine difference.
> > >
> > > If we look at the original staged pipeline, every single individual
> > > job took much longer than every individual jobs in the simplified
> > > pipeline. I think the difference in job times accounts for most
> > > (possibly all) of the difference in the pipelines time.
> > >
> > > If we look at the history of libvirt pipelines:
> > >
> > > https://gitlab.com/libvirt/libvirt/pipelines
> > >
> > > the vast majority of the time we're completing in 30 minutes or
> > > less already.
> That was before introducing FreeBSD builds, which for whatever reason
> take a significantly longer time: the last couple of jobs both took
> 50+ minutes. Installing packages is very inefficient, it would seem.
Oh dear, yeah, i missed that it introduced FreeBSD.
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
More information about the libvir-list