[libvirt PATCH 4/4] gitlab-ci: Introduce a new test 'integration' pipeline stage
Daniel P. Berrangé
berrange at redhat.com
Tue Feb 1 12:36:11 UTC 2022
On Mon, Jan 31, 2022 at 07:01:01PM +0100, Erik Skultety wrote:
> Create an integration child pipeline in this stage which will trigger a
> multi-project CI build of Perl bindings which are required by the TCK
> test suite.
> In general, this stage will install all the necessary build artifacts
> and configure logging on the worker node prior to executing the actual
> test suite. In case of a failure, libvirt and Avocado logs are saved
> and published as artifacts.
>
> Signed-off-by: Erik Skultety <eskultet at redhat.com>
> ---
> .gitlab-ci-integration.yml | 116 +++++++++++++++++++++++++++++++++++++
> .gitlab-ci.yml | 14 +++++
> 2 files changed, 130 insertions(+)
> create mode 100644 .gitlab-ci-integration.yml
>
> diff --git a/.gitlab-ci-integration.yml b/.gitlab-ci-integration.yml
> new file mode 100644
> index 0000000000..cabefc5166
> --- /dev/null
> +++ b/.gitlab-ci-integration.yml
> @@ -0,0 +1,116 @@
> +stages:
> + - bindings
> + - integration
> +
> +.tests:
> + stage: integration
> + before_script:
> + - mkdir "$SCRATCH_DIR"
> + - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/*
> + - sudo pip3 install --prefix=/usr avocado-framework
> + - source /etc/os-release # in order to query the vendor-provided variables
> + - if test "$ID" == "centos" && test "$VERSION_ID" -lt 9 ||
> + test "$ID" == "fedora" && test "$VERSION_ID" -lt 35;
> + then
> + DAEMONS="libvirtd virtlogd virtlockd";
> + else
> + DAEMONS="virtproxyd virtqemud virtinterfaced virtsecretd virtstoraged virtnwfilterd virtnodedevd virtlogd virtlockd";
> + fi
> + - for daemon in $DAEMONS;
> + do
> + sudo sed -Ei "s/^(#)?(log_outputs=).*/\2'1:file:\/var\/log\/libvirt\/${daemon}.log'/" /etc/libvirt/${daemon}.conf;
> + sudo sed -Ei "s/^(#)?(log_filters=).*/\2'4:*object* 4:*json* 4:*event* 4:*rpc* 4:daemon.remote 4:util.threadjob 4:*access* 1:*'/" /etc/libvirt/${daemon}.conf;
> + sudo systemctl --quiet stop ${daemon}.service;
> + sudo systemctl restart ${daemon}.socket;
> + done
> + - sudo virsh net-start default &>/dev/null || true;
What context is all this being run in ?
There's no docker image listed in the container, and I see the
'tags' referring to VMs. Is this really installing stuff as
root in the VM runnerrs ?
Basically I'm not seeing where any of this work is cleaned up
to give a pristine environment for the next pipeline, or what
happens if two pipelines run concurrently ?
> +
> + script:
> + - mkdir logs
> + - cd "$SCRATCH_DIR"
> + - git clone --depth 1 https://gitlab.com/libvirt/libvirt-tck.git
> + - cd libvirt-tck
> + - sudo avocado --config avocado.config run --job-results-dir "$SCRATCH_DIR"/avocado
> + after_script:
> + - if test -e "$SCRATCH_DIR"/avocado;
> + then
> + sudo mv "$SCRATCH_DIR"/avocado/latest/test-results logs/avocado;
> + fi
> + - sudo mv /var/log/libvirt logs/libvirt
> + - sudo chown -R $(whoami):$(whoami) logs
> + variables:
> + SCRATCH_DIR: "/tmp/scratch"
> + artifacts:
> + name: logs
> + paths:
> + - logs
> + when: on_failure
> +
> +
> +libvirt-perl-bindings:
> + stage: bindings
> + trigger:
> + project: eskultety/libvirt-perl
I assume pointing to this personal repo is a temp hack for some fix
that's not merged into main libvirt-perl ?
> + branch: multi-project-ci
> + strategy: depend
> +
> +
> +centos-stream-8-tests:
> + extends: .tests
> + needs:
> + - libvirt-perl-bindings
> + - pipeline: $PARENT_PIPELINE_ID
> + job: x86_64-centos-stream-8
> + - project: eskultety/libvirt-perl
> + job: x86_64-centos-stream-8
> + ref: multi-project-ci
> + artifacts: true
> + variables:
> + DISTRO: centos-stream-8
> + tags:
> + - centos-stream-vm
> +
> +centos-stream-9-tests:
> + extends: .tests
> + needs:
> + - libvirt-perl-bindings
> + - pipeline: $PARENT_PIPELINE_ID
> + job: x86_64-centos-stream-9
> + - project: eskultety/libvirt-perl
> + job: x86_64-centos-stream-9
> + ref: multi-project-ci
> + artifacts: true
> + variables:
> + DISTRO: centos-stream-9
> + tags:
> + - centos-stream-vm
> +
> +fedora-34-tests:
> + extends: .tests
> + needs:
> + - libvirt-perl-bindings
> + - pipeline: $PARENT_PIPELINE_ID
> + job: x86_64-fedora-34
> + - project: eskultety/libvirt-perl
> + job: x86_64-fedora-34
> + ref: multi-project-ci
> + artifacts: true
> + variables:
> + DISTRO: fedora-34
> + tags:
> + - fedora-vm
> +
> +fedora-35-tests:
> + extends: .tests
> + needs:
> + - libvirt-perl-bindings
> + - pipeline: $PARENT_PIPELINE_ID
> + job: x86_64-fedora-35
> + - project: eskultety/libvirt-perl
> + job: x86_64-fedora-35
> + ref: multi-project-ci
> + artifacts: true
> + variables:
> + DISTRO: fedora-35
> + tags:
> + - fedora-vm
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 4bcaf22ce2..453472c8be 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -4,6 +4,7 @@ variables:
> stages:
> - containers
> - builds
> + - test
> - sanity_checks
>
> .script_variables: &script_variables |
> @@ -128,3 +129,16 @@ coverity:
> - curl https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN --form email=$GITLAB_USER_EMAIL --form file=@cov-int.tar.gz --form version="$(git describe --tags)" --form description="$(git describe --tags) / $CI_COMMIT_TITLE / $CI_COMMIT_REF_NAME:$CI_PIPELINE_ID"
> rules:
> - if: "$CI_PIPELINE_SOURCE == 'schedule' && $COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN"
> +
> +integration:
> + stage: test
> + needs:
> + - x86_64-centos-stream-8
> + - x86_64-centos-stream-9
> + - x86_64-fedora-34
> + - x86_64-fedora-35
> + trigger:
> + include: .gitlab-ci-integration.yml
> + strategy: depend
> + variables:
> + PARENT_PIPELINE_ID: $CI_PIPELINE_ID
I've not really thought about the implications, so I'm curious what's
the rationale for using a separate pipeline in this way, as opposed to
have x86_64-fedora-35-integration, x86_64-centos-stream-9-integration,
etc jobs in the existing pipeline ?
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
More information about the libvir-list
mailing list