[Libguestfs] [libnbd RFC PATCH] First stab at CI infrastructure

[Richard W.M. Jones]

On Thu, Apr 15, 2021 at 10:09:48PM +0200, Martin Kletzander wrote:
> On Thu, Apr 08, 2021 at 05:19:25PM +0100, Richard W.M. Jones wrote:
> >On Thu, Apr 08, 2021 at 05:39:17PM +0200, Martin Kletzander wrote:
> >> - clang failures:
> >>   https://gitlab.com/nertpinx/libnbd/-/jobs/1163504401
> >
> >I don't usually enable warnings with clang, but I've pushed this which
> >fixes the problem above (and a further one) for me:
> >
> > https://gitlab.com/nbdkit/libnbd/-/commit/28816a5f8829f474d07351611140f461facd0e4d
> >
> 
> We can disable anything if you want.  Bear in mind this is a preparation
> for other projects, like nbdkit, so make sure you express all your
> desires so that I can adjust appropriately.  We can just keep clang
> running optionally or not enable warnings.

I think clang builds with warnings should be possible now.

> >> - golang test failures on OpenSUSE:
> >>   https://gitlab.com/nertpinx/libnbd/-/jobs/1163504416
> >
> >This is actually because:
> >
> > checking for nbdkit... no
> >
> 
> Oh, missed that.  There might be some extra dependencies missing and
> these are pain in the butt to search for.  But nbdkit is missing because
> we will want to do two things:
> 
>   - test it with the distro's nbdkit
> 
>   - test it with current nbdkit's master branch
> 
> I did not get to that yet, I have some preparation done, but we first
> need CI for nbdkit running properly and then it'll be easier to join
> them together.

There's a circular dependency between libnbd & nbdkit unfortunately
(although it's only required by tests).

> I added a lot of dependencies (what took a tremendous amount of time).
> Some are not added on all platforms, I'll be working on that, slowly.
> Even though the mapping from libvirt-ci helps a lot, the last 10% is
> still checking each missing dependency and skipped test on every
> distro. I am just about done with OpenSUSE Tumbleweed.
> 
> >but we go ahead and run the golang tests anyway.  Fixed here:
> >
> > https://gitlab.com/nbdkit/libnbd/-/commit/b15dde588f47894f9656ea62eb632156b7638b8b
> >
> >but there may be more problems after this, we'll see.
> >
> >> - one weird ubuntu 18.04 failure:
> >>   https://gitlab.com/nertpinx/libnbd/-/jobs/1163504418
> >
> >This has golang 1.10 which predates the inclusion of modules.  There's
> >basically no sane way to make libnbd work with both old and new
> >golang, so I would suggest --disable-golang on this platform.  (In
> >fact is it possible to disable features on different CI platforms?)
> 
> We can do anything!  If disabling golang is the best way to go (no pun
> intended), then we can do that.  Or disable it for any golang older than
> some 1.xx version.  For now I disabled golang on that old Ubuntu.

We need golang >= 1.13, so any platform that doesn't have that mininum
will need to use --disable-golang.  However golang does turn out to be
important (eg. for kubevirt) so we should enable it on the other
platforms.

> >>and if you want to be able to run `make distcheck` as well, there are
> >>some builddir/srcdir issues in golang/Makefile.am I believe, so VPATH
> >>building actually fails.  I added the `make distcheck` step to one of
> >>the jobs here:
> >>
> >>  https://gitlab.com/nertpinx/libnbd/-/jobs/1163504412
> >
> >I think "make distcheck" is a nice-to-have but ISTR that making it
> >work was a lot of work for libguestfs.
> >
> 
> If distcheck is not a priority, then we can just not run it. VPATH
> builds with autotools are hard and unless there is someone running
> VPATH builds all the time then they will break every now and then.
> 
> I'll keep the `make distcheck` guarded under `if $DISTCHECK == "force"`
> so that we can enable it later with one env var.

I think we can skip distcheck for now.  I only use 'make check'.

> >>In order not to bother you with every single e-mail I would suggest that
> >>I push some incomplete version in the repository and you can see
> >>yourself (and others as well) how the build behaves.  What do you say?
> >>I won't look at it earlier than tomorrow anyway.
> >
> >Sure, go for it.
> >
> 
> I added some more optional packages that were missing, so that most of
> the code is build/tested, changed a few more things and I'll at least
> make it consistent where possible, otherwise it would be a mess for
> anyone trying to make sense of it.  I hope that if I do it properly
> right away I will save some time for later.
> 
> There are still some issues, couple of them due to (probably just older)
> libev on centos-7 and ubuntu-18.04, FreeBSD not having fallocate() and
> some others that I have fixes for.  I'll let you know, thanks for the
> patience.

This all great, thanks a lot.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org