[Libguestfs] [libnbd PATCH 5/5] Introduction of automated testing

[Martin Kletzander]

On Fri, May 07, 2021 at 12:30:41PM -0500, Eric Blake wrote:
>On 5/6/21 6:30 AM, Martin Kletzander wrote:
>> This setup uses the libvirt-ci to get as much coverage as possible with the ease
>> of use provided by that repository.  Not all the data are available there at the
>> time of posting this patch, so if anyone wants to recreate the Containerfiles
>> and variable files (for cirrus CI) my temporary branch of libvirt-ci called
>> nbd_prep has all the work that is going to be proposed for inclusion there once
>> this goes in:
>>
>>   https://gitlab.com/nertpinx/libvirt-ci/-/tree/nbd_prep
>>
>> The result of this branch CI run is available here:
>>
>>   https://gitlab.com/nertpinx/libnbd/-/pipelines
>>
>> If there are any questions related to how the CI is running, how it
>> works, how to replicate CI builds locally or how to change anything,
>> then I am more than happy to help.
>>
>> Actually recreating the builds locally (at least for Linux distributions
>> and setups) is pretty straightforward. Choose a file from ci/containers
>> which represents the desired setup, for our example let's pick fedora
>> rawhide, and build your container and tag it, e.g. using podman (or feel
>> free to substitute "podman" with "docker"):
>>
>>   podman build ci/containers/fedora-33.Dockerfile -t libnbd-fedora-rawhide
>
>Gave me an error message:
>Error: context must be a directory:
>"/home/eblake/libnbd/ci/containers/ubuntu-1804.Dockerfile"
>
>I'm assuming you meant 'podman build -f ci/...'
>

Yep, it's missing `-f` in the command.

>>
>> That will get you a container tagged `libnbd-fedora-rawhide` that you
>> can execute the tests on.  You can then run whatever you want inside
>> that container with the current repository passed through like this:
>>
>>   podman run -it --rm -v .:/repo -w /repo libnbd-fedora-rawhide bash
>
>I'm getting:
>root at ccd5058d1cf8:/repo# cd /repo
>root at ccd5058d1cf8:/repo# ls
>ls: cannot open directory '.': Permission denied
>root at ccd5058d1cf8:/repo# cd /
>root at ccd5058d1cf8:/# ls /repo
>ls: cannot open directory '/repo': Permission denied
>
>when running that command line as a non-root user in the host.  Am I
>missing a setup step?
>

It might be SELinux related, the command might need something like:

   -v .:/repo:Z

instead, so that it is relabelled properly.

Or maybe, but very improbably,

   -v .:/repo:Z,U

I did not have to use that on non-SELinux system so I guess the U flag
is not needed.

>>
>> which will bind-mount the current directory onto /repo inside the
>> container and also use that path as the working directory (just so you
>> do not have to `cd /repo` before any commands.  I prefer running bash,
>> but of course you can just run the build script used in the CI.  I have
>> put all the CI build commands into one file for simplicity, so that you
>> can simply run `ci/build_script.sh`.  Executing that script will give
>> you the results and you can experiment right inside that environment to
>> figure out what is needed.  At the same time you can easily modify any
>> files inside that repository on your host, just like you are used to, so
>> that you can use your editor and other setups that work for you.  Just
>> make sure that build files do not interfere between the host and the
>> container, if you want to replicate a clean build you need to either use
>> VPATH or just clean everything.
>>
>> Let me know what you think, and have a nice day.
>>
>
>> +++ b/ci/build_script.sh
>> @@ -0,0 +1,43 @@
>> +#!/bin/sh
>> +
>> +set -e
>> +
>> +main() {
>> +    autoreconf -if
>> +
>> +    CONFIG_ARGS="--enable-gcc-warnings --enable-fuse --enable-ocaml --enable-python --enable-golang --with-gnutls --with-libxml2"
>
>Long line; worth splitting with \-newline?
>
>> +
>> +    if test "$GOLANG" != "skip"
>> +    then
>> +       CONFIG_ARGS="$CONFIG_ARGS --enable-golang"
>> +    fi
>> +
>> +    ./configure $CONFIG_ARGS
>> +
>> +    $MAKE
>> +
>> +    if test -n "$CROSS"
>> +    then
>> +       return 0
>> +    fi
>> +
>> +    $MAKE check || { find . -name test-suite.log -exec grep -l '^X\?FAIL:' '{}' \+ | xargs cat; exit 1; }
>
>and another
>
>> +
>> +    if test "$CHECK_VALGRIND" = "force"
>> +    then
>> +        $MAKE check-valgrind
>> +    fi
>> +
>> +    if test "$DIST" != "skip"
>> +    then
>> +        $MAKE dist
>> +        $MAKE maintainer-check-extra-dist
>> +    fi
>> +
>> +    if test "$DISTCHECK" = "force"
>> +    then
>> +        $MAKE distcheck
>> +    fi
>> +}
>> +
>> +main "$@"
>
>Didn't see any bashisms, so keeping this /bin/sh seems fine.
>

I tried.  I will split the long lines, that's a good point.

>
>> +++ b/ci/containers/centos-8.Dockerfile
>> @@ -0,0 +1,61 @@
>> +# THIS FILE WAS AUTO-GENERATED
>> +#
>
>Do we really want to check in auto-generated files into git?
>

The Dockerfiles need to be present in the repository so that the CI can
use it to run the build and tests.  It also makes it possible for anyone
to reproduce or check the builds locally without the need for anything
else.  The generator is basically just a helper to update this file.
This is the same way that all the other projects using libvirt-ci work.

>> +#  $ lcitool dockerfile centos-8 libnbd
>> +#
>> +# https://gitlab.com/libvirt/libvirt-ci/-/commit/684ac15bdf4de9c5b07614f9bbf20ab0815a92c2
>
>You do have a point, however, that libvirt-ci (and in particular, a VERY
>recent non-released commit) is not standard on other machines, so
>checking them in makes life easier.
>

Once the libvirt-CI patches are in I will submit the libvirt-ci patches
upstream.  I first want to make sure they are correct.

>--
>Eric Blake, Principal Software Engineer
>Red Hat, Inc.           +1-919-301-3226
>Virtualization:  qemu.org | libvirt.org
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20210512/12dfae79/attachment.sig>