[Avocado-devel] Avocado testplans
Sergey Bronnikov
estetus at gmail.com
Fri Aug 3 08:20:28 UTC 2018
Lukas, thanks for explanation! But I still have a question.
Imagine we have an YAML config with a list of tests and their parameters:
```yaml
basic: !mux
test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
timeout: 360
test_reference_resolver_args: !!python/dict
vt_extra_params:
- nic_model=rtl8139
- extra_params=" -cpu host"
- drive_format=ahci
- mem=2048
vt-machine-type: q35
shutdown:
test_reference: io-github-autotest-qemu.shutdown
vt_extra_params:
- shutdown_method = shell
reboot:
test_reference: io-github-autotest-qemu.reboot
vt_extra_params:
- param1=1
- param2=2
stress:
test_reference: io-github-autotest-qemu.linux_stress
iofuzz:
test_reference: io-github-autotest-qemu.iofuzz
vt_extra_params:
- param1=1
- param2=2
```
How to execute tests from this testplan?
$ avocado run -m basic.yaml
No test references provided nor any other arguments resolved into tests.
Please double check the executed command.
Sergey
чт, 2 авг. 2018 г. в 17:08, Lukáš Doktor <ldoktor at redhat.com>:
> Dne 1.8.2018 v 13:19 Sergey Bronnikov napsal(a):
> > Hello!
> >
> > I'm trying to find a way to organize my tests for running without
> > specifying all of them in command-line. Imagine we have a set of
> different
> > tests: sleep.py, passtest.py and fail.py. To run all of them with
> required
> > parameters (aka variants) we may execute a command:
> > avocado run sleep.py fail.py passtest.py -m basic.yaml
> >
> > Is there a way to run the same tests without explicit specifying them?
> > For example: avocado run -m basic.yaml
> >
> > Sergey
> >
>
> Dear Sergey,
>
> as a matter of fact, there are multiple ways.
>
> 1. filter-by-tag:
> http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#categorizing-tests
>
> you can tag your tests eg. `sanity`, `release`, ... and then run
> `avocado run --filter-by-tag=sanity -- TEST_REPO` to only run tests tagged
> `sanity` (note the granularity is per-class or even per-test-method.
>
> 2. yaml loader optional plugin:
> http://avocado-framework.readthedocs.io/en/latest/optional_plugins/yaml_loader.html
>
> which is probably more what you're looking for, but currently only
> works on python2 (but should get py3 support, hopefully, soon). It works
> similarly to yaml_to_mux varianter plugin, but allows special keys like
> `test_reference` to specify what test will be used or even
> `test_reference_resolver_class` when your tests are not the default ones.
> This is for example useful when you want to include multiple
> external-runner tests with different external-runner. There are examples
> in:
> https://github.com/avocado-framework/avocado/tree/master/examples/yaml_to_mux_loader
> but let me share my slighly advanced example here:
>
> ```yaml
> # Generated s390x mux-suite
> !mux
> qemu_make: !mux
> mux_suite_test_name_prefix: 'qemu_make/git '
> check:
> test_reference:
> /home/jenkins/s390x/qemu-master/avocado-qemu-make-check
> install:
> test_reference:
> /home/jenkins/s390x/qemu-master/avocado-qemu-make-install
>
> unit:
> variants: !mux
> test_reference_resolver_class: avocado.core.loader.ExternalLoader
> timeout: 360
> rpm-tcg:
> mux_suite_test_name_prefix: 'unit/rpm/tcg '
> test_reference_resolver_extra: !!python/dict
> loader_options:
> /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-tcg
> rpm-kvm:
> mux_suite_test_name_prefix: 'unit/rpm/kvm '
> test_reference_resolver_extra: !!python/dict
> loader_options:
> /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-kvm
> git-tcg:
> mux_suite_test_name_prefix: 'unit/git/tcg '
> test_reference_resolver_extra: !!python/dict
> loader_options:
> /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-tcg
> git-kvm:
> mux_suite_test_name_prefix: 'unit/git/kvm '
> test_reference_resolver_extra: !!python/dict
> loader_options:
> /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-kvm
> tests: !mux
> cmm:
> test_reference: cmm
> diag10:
> test_reference: diag10
> emulator:
> test_reference: emulator
> gs:
> test_reference: gs
> iep:
> test_reference: iep
> intercept:
> test_reference: intercept
> pfmf:
> test_reference: pfmf
> selftest-setup:
> test_reference: selftest-setup
> sieve:
> test_reference: sieve
> skey:
> test_reference: skey
> sthyi:
> test_reference: sthyi
> vector:
> test_reference: vector
> functional: !mux
> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
> s390x-rpm-libvirt/dvd:
> mux_suite_test_name_prefix: 'functional/rpm/libvirt/dvd '
> test_reference:
> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
> io-github-autotest-qemu.boot remove_guest.without_disk
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_type: libvirt
> vt_extra_params:
> - automem=no
> - url=$URL
> - vga=none
> s390x-rpm-libvirt/url:
> mux_suite_test_name_prefix: 'functional/rpm/libvirt/url '
> test_reference:
> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
> io-github-autotest-qemu.boot remove_guest.without_disk
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_type: libvirt
> vt_extra_params:
> - automem=no
> - url=$URL
> - vga=none
> s390x-rpm-dvd:
> mux_suite_test_name_prefix: 'functional/rpm/dvd '
> test_reference:
> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads boot
> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> migrate.default.tcp.with_post_copy
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_extra_params:
> - automem=no
> - url=$URL
> - ping_pong=5
> - vga=none
> s390x-rpm-url:
> mux_suite_test_name_prefix: 'functional/rpm/url '
> test_reference:
> unattended_install.url.extra_cdrom_ks.default_install.aio_threads boot
> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> migrate.default.tcp.with_post_copy
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_extra_params:
> - automem=no
> - url=$URL
> - ping_pong=5
> - vga=none
> s390x-git-dvd:
> mux_suite_test_name_prefix: 'functional/git/dvd '
> test_reference:
> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads boot
> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> migrate.default.tcp.with_post_copy
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin:
> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> vt_dst_qemu_bin:
> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> vt_extra_params:
> - automem=no
> - url=$URL
> - ping_pong=5
> - vga=none
> s390x-git-url:
> mux_suite_test_name_prefix: 'functional/git/url '
> test_reference:
> unattended_install.url.extra_cdrom_ks.default_install.aio_threads boot
> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> migrate.default.tcp.with_post_copy
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin:
> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> vt_dst_qemu_bin:
> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> vt_extra_params:
> - automem=no
> - url=$URL
> - ping_pong=5
> - vga=none
> ```
>
> (feel free to ask for details, but basically it uses SIMPLE tests,
> EXTERNAL_RUNNER tests and then Avocado-vt tests. The test always receives
> all the params so you can override the standard arguments. If you specify
> "test_reference_resolver_args" it also updates the loader parameters, which
> is what I use for Avocado-vt tests).
>
> Theoretically there is yet another way and that is to create directory
> "sanity" and put symlinks to actual tests there, but the granularity there
> would be only per-test-class, which usually does not suffice.
>
> Anyway, hopefully at least one method suits you, but feel free to
> elaborate more, we can always improve.
>
> Kind regards,
> Lukáš
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20180803/04e55f2e/attachment.htm>
More information about the Avocado-devel
mailing list