<div dir="ltr"><div>Lukas, thanks for explanation! But I still have a question.</div><div> </div><div>Imagine we have an YAML config with a list of tests and their parameters: </div><div> </div><div>```yaml</div><div>basic: !mux test_reference_resolver_class: avocado_vt.loader.VirtTestLoader timeout: 360 test_reference_resolver_args: !!python/dict vt_extra_params: - nic_model=rtl8139 - extra_params=" -cpu host" - drive_format=ahci - mem=2048 vt-machine-type: q35 shutdown: test_reference: io-github-autotest-qemu.shutdown vt_extra_params: - shutdown_method = shell reboot: test_reference: io-github-autotest-qemu.reboot vt_extra_params: - param1=1 - param2=2 stress: test_reference: io-github-autotest-qemu.linux_stress iofuzz: test_reference: io-github-autotest-qemu.iofuzz vt_extra_params: - param1=1 - param2=2 </div><div>```</div><div> </div><div>How to execute tests from this testplan? </div><div> </div><div>$ avocado run -m basic.yaml No test references provided nor any other arguments resolved into tests. Please double check the executed command. </div><div> </div><div>Sergey </div><div> </div><div> </div></div> <div class="gmail_quote"><div dir="ltr">чт, 2 авг. 2018 г. в 17:08, Lukáš Doktor <<a href="mailto:ldoktor@redhat.com">ldoktor@redhat.com</a>>: </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dne 1.8.2018 v 13:19 Sergey Bronnikov napsal(a): > Hello! > > I'm trying to find a way to organize my tests for running without > specifying all of them in command-line. Imagine we have a set of different > tests: sleep.py, passtest.py and fail.py. To run all of them with required > parameters (aka variants) we may execute a command: > avocado run sleep.py fail.py passtest.py -m basic.yaml > > Is there a way to run the same tests without explicit specifying them? > For example: avocado run -m basic.yaml > > Sergey > Dear Sergey, as a matter of fact, there are multiple ways. 1. filter-by-tag: <a href="http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#categorizing-tests" rel="noreferrer" target="_blank">http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#categorizing-tests</a> you can tag your tests eg. `sanity`, `release`, ... and then run `avocado run --filter-by-tag=sanity -- TEST_REPO` to only run tests tagged `sanity` (note the granularity is per-class or even per-test-method. 2. yaml loader optional plugin: <a href="http://avocado-framework.readthedocs.io/en/latest/optional_plugins/yaml_loader.html" rel="noreferrer" target="_blank">http://avocado-framework.readthedocs.io/en/latest/optional_plugins/yaml_loader.html</a> which is probably more what you're looking for, but currently only works on python2 (but should get py3 support, hopefully, soon). It works similarly to yaml_to_mux varianter plugin, but allows special keys like `test_reference` to specify what test will be used or even `test_reference_resolver_class` when your tests are not the default ones. This is for example useful when you want to include multiple external-runner tests with different external-runner. There are examples in: <a href="https://github.com/avocado-framework/avocado/tree/master/examples/yaml_to_mux_loader" rel="noreferrer" target="_blank">https://github.com/avocado-framework/avocado/tree/master/examples/yaml_to_mux_loader</a> but let me share my slighly advanced example here: ```yaml # Generated s390x mux-suite !mux qemu_make: !mux mux_suite_test_name_prefix: 'qemu_make/git ' check: test_reference: /home/jenkins/s390x/qemu-master/avocado-qemu-make-check install: test_reference: /home/jenkins/s390x/qemu-master/avocado-qemu-make-install unit: variants: !mux test_reference_resolver_class: avocado.core.loader.ExternalLoader timeout: 360 rpm-tcg: mux_suite_test_name_prefix: 'unit/rpm/tcg ' test_reference_resolver_extra: !!python/dict loader_options: /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-tcg rpm-kvm: mux_suite_test_name_prefix: 'unit/rpm/kvm ' test_reference_resolver_extra: !!python/dict loader_options: /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-kvm git-tcg: mux_suite_test_name_prefix: 'unit/git/tcg ' test_reference_resolver_extra: !!python/dict loader_options: /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-tcg git-kvm: mux_suite_test_name_prefix: 'unit/git/kvm ' test_reference_resolver_extra: !!python/dict loader_options: /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-kvm tests: !mux cmm: test_reference: cmm diag10: test_reference: diag10 emulator: test_reference: emulator gs: test_reference: gs iep: test_reference: iep intercept: test_reference: intercept pfmf: test_reference: pfmf selftest-setup: test_reference: selftest-setup sieve: test_reference: sieve skey: test_reference: skey sthyi: test_reference: sthyi vector: test_reference: vector functional: !mux test_reference_resolver_class: avocado_vt.loader.VirtTestLoader s390x-rpm-libvirt/dvd: mux_suite_test_name_prefix: 'functional/rpm/libvirt/dvd ' test_reference: unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads io-github-autotest-qemu.boot remove_guest.without_disk test_reference_resolver_args: !!python/dict vt_qemu_bin: /usr/libexec/qemu-kvm vt_dst_qemu_bin: /usr/libexec/qemu-kvm vt_type: libvirt vt_extra_params: - automem=no - url=$URL - vga=none s390x-rpm-libvirt/url: mux_suite_test_name_prefix: 'functional/rpm/libvirt/url ' test_reference: unattended_install.url.extra_cdrom_ks.default_install.aio_threads io-github-autotest-qemu.boot remove_guest.without_disk test_reference_resolver_args: !!python/dict vt_qemu_bin: /usr/libexec/qemu-kvm vt_dst_qemu_bin: /usr/libexec/qemu-kvm vt_type: libvirt vt_extra_params: - automem=no - url=$URL - vga=none s390x-rpm-dvd: mux_suite_test_name_prefix: 'functional/rpm/dvd ' test_reference: unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads boot migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec migrate.default.tcp.with_post_copy test_reference_resolver_args: !!python/dict vt_qemu_bin: /usr/libexec/qemu-kvm vt_dst_qemu_bin: /usr/libexec/qemu-kvm vt_extra_params: - automem=no - url=$URL - ping_pong=5 - vga=none s390x-rpm-url: mux_suite_test_name_prefix: 'functional/rpm/url ' test_reference: unattended_install.url.extra_cdrom_ks.default_install.aio_threads boot migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec migrate.default.tcp.with_post_copy test_reference_resolver_args: !!python/dict vt_qemu_bin: /usr/libexec/qemu-kvm vt_dst_qemu_bin: /usr/libexec/qemu-kvm vt_extra_params: - automem=no - url=$URL - ping_pong=5 - vga=none s390x-git-dvd: mux_suite_test_name_prefix: 'functional/git/dvd ' test_reference: unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads boot migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec migrate.default.tcp.with_post_copy test_reference_resolver_args: !!python/dict vt_qemu_bin: /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x vt_dst_qemu_bin: /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x vt_extra_params: - automem=no - url=$URL - ping_pong=5 - vga=none s390x-git-url: mux_suite_test_name_prefix: 'functional/git/url ' test_reference: unattended_install.url.extra_cdrom_ks.default_install.aio_threads boot migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec migrate.default.tcp.with_post_copy test_reference_resolver_args: !!python/dict vt_qemu_bin: /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x vt_dst_qemu_bin: /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x vt_extra_params: - automem=no - url=$URL - ping_pong=5 - vga=none ``` (feel free to ask for details, but basically it uses SIMPLE tests, EXTERNAL_RUNNER tests and then Avocado-vt tests. The test always receives all the params so you can override the standard arguments. If you specify "test_reference_resolver_args" it also updates the loader parameters, which is what I use for Avocado-vt tests). Theoretically there is yet another way and that is to create directory "sanity" and put symlinks to actual tests there, but the granularity there would be only per-test-class, which usually does not suffice. Anyway, hopefully at least one method suits you, but feel free to elaborate more, we can always improve. Kind regards, Lukáš </blockquote></div>