[Avocado-devel] Avocado testplans
Lukáš Doktor
ldoktor at redhat.com
Wed Aug 8 09:06:12 UTC 2018
Dne 7.8.2018 v 19:26 Sergey Bronnikov napsal(a):
> Hello!
>
> Finally it works! You are right - the problem was due to absence of plugin
> 'loader_yaml'.
>
Glad to hear that. As usual if you happen to find some time to review our documentation, please do so so next users have less difficulties setting things up.
> I have another two questions:
> 1. How to use --vt-extra-params in command line and vt_extra_params section
> in YAML at the same time? Option kernel is ignored when I specify it in
> 'avocado run config.yaml --vt-extra-params kernel=path_to_kernel'.
>
> config.yaml:
>
> basic: !mux
> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
> timeout: 360
> test_reference_resolver_args: !!python/dict
> vt_machine_type: q35
> vt_extra_params:
> - extra_params=" -cpu host"
> - nic_model=rtl8139
> - drive_format=ahci
> - mem=2048
>
> shutdown:
> test_reference: io-github-autotest-qemu.shutdown
> test_reference_resolver_extra:
> avocado_vt_extra_params:
> - shutdown_method = shell
>
> How to avoid it?
Currently this is not possible, because the "test_reference_resolver_args" are treated is:
_args = params.get("test_reference_resolver_args")
if not _args:
args = self.args
else:
args = copy.deepcopy(self.args)
for key, value in iteritems(_args):
setattr(args, key, value)
so basically args from mux-suite replaces arg set on the command line. Using only "test_reference_resolver_extra" should work as there we do:
if vt_extra_params:
# We don't want to override the original args
self.args = copy.deepcopy(self.args)
if getattr(self.args, 'vt_extra_params', None) is not None:
self.args.vt_extra_params += vt_extra_params
else:
self.args.vt_extra_params = vt_extra_params
but unfortunately (as far as I know) there is no way on combining default extra-params and extra-params from variant (you'd have to set all of them everywhere, which would be probably a bit odd).
Anyway if you had an idea on how to do this that would work generally and not just for avocado-vt, please let us know.
>
> 2. What is a proper way to specify guest os in YAML config?
> I want to make a separate YAML config with guest OSes and mux it with test
> config inserted above. As far as I understand it is need to somehow
> override vt_guest option or use specific parameter from guest OS configs
> (like image_name or os_variant) in YAML. But vt_guest_os is not processed
> in avocado_vt/loader.py and second variant doesn't work too. Is is ever
> possible without touching source code?
>
And here I'm afraid I also can't help much. The mux-suite only allows specifying a single file so in the end you have to merge the files.
Even if we had, we don't allow to combine args from multiple sources (RFCs about that are welcome). The only way I can imagine is to define the overall "test_reference_resolver_args" in the OS variant mux-suite and then create multiple mux-suite files with your testsuites. Then before running the testsuite you could do:
```shell
cat variants.yaml > to_be_executed.yaml
cat basic.yaml >> to_be_executed.yaml
avocado run to_be_executed.yaml
```
where "variants.yaml" would look like this:
```yaml
variants: !mux
test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
timeout: 360
q35-rhel-7:
test_reference_resolver_args: !!python/dict
vt_extra_params:
- mem=2048
- nic_model=rtl8139
- drive_format=ahci
- disk_bus=ahci
- extra_params=" -cpu host"
vt_machine_type: q35
vt_guest_os: RHEL.7.devel
i440fx-rhel-7:
test_reference_resolver_args: !!python/dict
vt_extra_params:
- mem=2048
- nic_model=rtl8139
- drive_format=ahci
- disk_bus=ahci
- extra_params=" -cpu host"
vt_machine_type: i440fx
vt_guest_os: RHEL.7.devel
arm64-JeOS-7:
test_reference_resolver_args: !!python/dict
vt_extra_params:
- mem=2048
- nic_model=rtl8139
- drive_format=ahci
- disk_bus=ahci
- extra_params=" -cpu host"
vt_machine_type: arm64-pci
vt_guest_os: JeOS.27
s390x-Fedora-27:
test_reference_resolver_args: !!python/dict
vt_extra_params:
- mem=2048
- nic_model=rtl8139
- drive_format=ahci
- disk_bus=ahci
- extra_params=" -cpu host"
vt_machine_type: s390-virtio
vt_guest_os: Fedora-27
```
and the `basic.yaml` must not use "test_reference_resolver_args". (note the "test_reference_resolver_class" could be omitted in `variants.yaml` in case you need non-avocado-vt tests in the basic.yaml as well). So it'll look somehow like this:
```yaml
basic: !mux
shutdown:
test_reference: io-github-autotest-qemu.shutdown
test_reference_resolver_extra: !!python/dict
avocado_vt_extra_params:
- shutdown_method = shell
reboot:
test_reference: io-github-autotest-qemu.reboot
test_reference_resolver_extra: !!python/dict
avocado_vt_extra_params:
- param1 = 1
- param2 = 2
stress:
test_reference: io-github-autotest-qemu.linux_stress
iofuzz:
test_reference: io-github-autotest-qemu.iofuzz
test_reference_resolver_extra: !!python/dict
avocado_vt_extra_params:
- param1 = 1
- param2 = 2
```
It's not perfect and you have to define all the `resolver_args` in one place, but it should work.
Btw this does not affect me because I have my "mux-suites" generated from bash. They are usually more verbose than a human would write, but they allow me to combine different types of tests and different variants (my usage is usually different qemu versions, some simple/instrumented tests and some avocado-vt tests)
Regards,
Lukáš
> Sergey
>
> вт, 7 авг. 2018 г. в 12:59, Lukáš Doktor <ldoktor at redhat.com>:
>
>> Dne 7.8.2018 v 11:16 Sergey Bronnikov napsal(a):
>>> пн, 6 авг. 2018 г. в 19:03, Lukáš Doktor <ldoktor at redhat.com>:
>>>
>>>> Dne 6.8.2018 v 09:52 Sergey Bronnikov napsal(a):
>>>>> пт, 3 авг. 2018 г. в 14:55, Lukáš Doktor <ldoktor at redhat.com>:
>>>>>
>>>>>> Well the execution is actually simpler than you think, `avocado run
>>>>>> basic.yaml` should does the trick. Anyway the yaml-file is slightly
>>>>>> corrupted and even then some things won't probably result in what you
>>>>>> expect. Let me explain it a bit and provide better example)
>>>>>>
>>>>>> Dne 3.8.2018 v 10:20 Sergey Bronnikov napsal(a):
>>>>>>> Lukas, thanks for explanation! But I still have a question.
>>>>>>>
>>>>>>> Imagine we have an YAML config with a list of tests and their
>>>> parameters:
>>>>>>>
>>>>>>> ```yaml
>>>>>>> basic: !mux
>>>>>>>
>>>>>>> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
>>>>>>> timeout: 360
>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>> vt_extra_params:
>>>>>>> - nic_model=rtl8139
>>>>>>> - extra_params=" -cpu host"
>>>>>>> - drive_format=ahci
>>>>>>> - mem=2048
>>>>>>> vt-machine-type: q35
>>>>>>
>>>>>> this should be "vt_machine_type"
>>>>>>
>>>>>>>
>>>>>>> shutdown:
>>>>>>> test_reference: io-github-autotest-qemu.shutdown
>>>>>>
>>>>>> Here `test_reference_resolver_extra:` is missing.
>>>>>>
>>>>>>> vt_extra_params:
>>>>>>
>>>>>> this should be `avocado_vt_extra_params:` (see `avocado_vt/loader.py`)
>>>>>>
>>>>>>> - shutdown_method = shell
>>>>>>>
>>>>>>> reboot:
>>>>>>> test_reference: io-github-autotest-qemu.reboot
>>>>>>
>>>>>> Here `test_reference_resolver_extra:` is missing.
>>>>>>
>>>>>>> vt_extra_params:
>>>>>>> - param1=1
>>>>>>> - param2=2
>>>>>>>
>>>>>>> stress:
>>>>>>> test_reference: io-github-autotest-qemu.linux_stress
>>>>>>>
>>>>>>> iofuzz:
>>>>>>> test_reference: io-github-autotest-qemu.iofuzz
>>>>>>
>>>>>> Here `test_reference_resolver_extra:` is missing.
>>>>>>
>>>>>>> vt_extra_params:
>>>>>>> - param1=1
>>>>>>> - param2=2
>>>>>>>
>>>>>>> ```
>>>>>>>
>>>>>>> How to execute tests from this testplan?
>>>>>>
>>>>>> as mentioned earlier, it's recognized as test format, therefor simply
>>>>>> `avocado run basic.yaml`.
>>>>>>
>>>>>> Also one thing, the vt discovery is quite slow (yeah we parse
>> cartesian
>>>>>> config on each invocation as it might have different values). You can
>>>> put
>>>>>> multiple `test_reference` into the same line and separate them by
>> space
>>>> in
>>>>>> avocado-vt loader. But beware, it works as filters, therefor the
>>>> discovered
>>>>>> tests will be re-oredered accordingly to cartesian config (rather than
>>>> how
>>>>>> you put them there) and they'll occur only once. Therefor:
>>>>>>
>>>>>> test_reference: shutdown boot boot
>>>>>>
>>>>>> will generate:
>>>>>>
>>>>>> avocado list basic.yaml
>>>>>> VT io-github-autotest-qemu.boot
>>>>>> VT io-github-autotest-qemu.shutdown
>>>>>>
>>>>>> Anyway the speed of that is incomparably faster, which is why I use it
>>>>>> when possible.
>>>>>>
>>>>>
>>>>>
>>>>> Finally with corrections I got a following testplan:
>>>>
>>>> Dear Sergey,
>>>>
>>>> actually you unveiled a bug; fix can be found here:
>>>> https://github.com/avocado-framework/avocado-vt/pull/1696 (in my CI I
>>>> always use `--vt-extra-params` so I forgot it might not be set. Sorry
>> about
>>>> this).
>>>>
>>>>
>>> `avocado list plan.yaml` still fails. With --verbose it awlays shows
>> error
>>> for any testplan:
>>> "Unexpected character / on pos 7. Special chars are allowed only in
>>> variable assignation statement: 'only ../avocado-vt/plans/lukas.yaml'
>>> (<string>:1)"
>>>
>>> Common error for any YAML testplan:
>>> ~$ avocado list --verbose ../avocado-vt/plans/lukas.yaml
>>> Type
>>> Test
>>> Tag(s)
>>> NOT_A_TEST ../avocado-vt/plans/lukas.yaml: Not an INSTRUMENTED
>>> (avocado.Test based), PyUNITTEST (unittest.TestCase based) or SIMPLE
>>> (executable) test
>>> !VT ../avocado-vt/plans/lukas.yaml: Unexpected character / on pos
>>> 7. Special chars are allowed only in variable assignation statement:
>> 'only
>>> ../avocado-vt/plans/lukas.yaml' (<string>:1)
>>>
>>> TEST TYPES SUMMARY
>>> ...
>>>
>>> ~$ curl
>>>
>> https://github.com/avocado-framework/avocado/blob/master/examples/yaml_to_mux_loader/basic.yaml
>>
>> I assume you used `curl
>> https://raw.githubusercontent.com/avocado-framework/avocado/master/examples/yaml_to_mux_loader/basic.yaml`
>> <https://raw.githubusercontent.com/avocado-framework/avocado/master/examples/yaml_to_mux_loader/basic.yaml>
>>
>>>> ../avocado-vt/plans/basic.yaml
>>> ~$ avocado list --verbose ../avocado-vt/plans/basic.yaml
>>> Type
>>> Test
>>> Tag(s)
>>> NOT_A_TEST ../avocado-vt/plans/basic.yaml: Not an INSTRUMENTED
>>> (avocado.Test based), PyUNITTEST (unittest.TestCase based) or SIMPLE
>>> (executable) test
>>> !VT ../avocado-vt/plans/basic-plain.yaml: Unexpected character /
>> on
>>> pos 7. Special chars are allowed only in variable assignation statement:
>>> 'only ../avocado-vt/plans/basic-plain.yaml' (<string>:1)
>>>
>>
>> This is correct, it's NOT_A_TEST and !VT. Anyway there should be the
>> "loader_yaml" installed which should understand that format and discover
>> the tests defined by the varianter. What is the output of:
>>
>> $ avocado list --loaders help
>> Unknown loader 'help'. Available plugins are:
>> glib: glib.GLIB
>> golang: golang.GOLANG
>> yaml_testsuite:
>> robot: robot.ROBOT
>> vt: vt.VT
>> file: file.SIMPLE, file.INSTRUMENTED, file.PyUNITTEST
>> external: external.EXTERNAL
>>
>> If the "yaml_testsuite is not there you need to install it (cd
>> optional_plugins/loader_yaml && python setup.py install). If you use `make
>> link` it should be there but `make install` won't install the optional
>> plugins.
>>
>> $ avocado list --verbose examples/mux-lukas.yaml
>> Type Test
>>
>> Tag(s)
>> NOT_A_TEST examples/mux-lukas.yaml: Not an INSTRUMENTED (avocado.Test
>> based), PyUNITTEST (unittest.TestCase based) or SIMPLE (executable) test
>> !GLIB examples/mux-lukas.yaml: No GLib-like tests found
>> !GOLANG examples/mux-lukas.yaml: Go binary not found.
>> INSTRUMENTED passtest.py:PassTest.test
>>
>> fast
>> INSTRUMENTED sleeptest.py:SleepTest.test
>> INSTRUMENTED failtest.py:FailTest.test
>>
>> failure_expected
>> INSTRUMENTED failtest.py:FailTest.test
>>
>> failure_expected
>> INSTRUMENTED failtest.py:FailTest.test
>>
>> failure_expected
>> !ROBOT examples/mux-lukas.yaml: Unsupported file format 'yaml'.
>> !VT examples/mux-lukas.yaml: Unexpected character / on pos 13.
>> Special chars are allowed only in variable assignation statement: 'only
>> examples/mux-lukas.yaml' (<string>:1)
>>
>> $ avocado list examples/mux-lukas.yaml
>> INSTRUMENTED passtest.py:PassTest.test
>> INSTRUMENTED sleeptest.py:SleepTest.test
>> INSTRUMENTED failtest.py:FailTest.test
>> INSTRUMENTED failtest.py:FailTest.test
>> INSTRUMENTED failtest.py:FailTest.test
>>
>> Hopefully it's just the missing plugin,
>> Lukáš
>>
>>
>>> TEST TYPES SUMMARY
>>> ==================
>>> !VT: 1
>>> ACCESS_DENIED: 0
>>> BROKEN_SYMLINK: 0
>>> EXTERNAL: 0
>>> INSTRUMENTED: 0
>>> MISSING: 0
>>> NOT_A_TEST: 1
>>> PYUNITTEST: 0
>>> SIMPLE: 0
>>> VT: 0
>>>
>>> Sergey
>>>
>>>
>>>> With this there is still one little remaining issue in your
>> `basic.yaml`,
>>>> see below...
>>>>
>>>>>
>>>>> basic: !mux
>>>>> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
>>>>> timeout: 360
>>>>> test_reference_resolver_extra: !!python/dict
>>>>
>>>> This is supposed to be "test_reference_resolver_args" (you're modifying
>>>> the `self.args` of Avocado for the purpose of this loader. Note this
>> only
>>>> affects things inside the loader, so you can't for example change the
>>>> job-results-dir or so...)
>>>>
>>>>> vt_extra_params:
>>>>> - mem=2048
>>>>> - nic_model=rtl8139
>>>>> - drive_format=ahci
>>>>> - disk_bus=ahci
>>>>> - extra_params=" -cpu host"
>>>>> vt_machine_type: q35
>>>>>
>>>>> shutdown:
>>>>> test_reference: io-github-autotest-qemu.shutdown
>>>>> test_reference_resolver_extra: !!python/dict
>>>>> avocado_vt_extra_params:
>>>>> - shutdown_method = shell
>>>>>
>>>>> reboot:
>>>>> test_reference: io-github-autotest-qemu.reboot
>>>>> test_reference_resolver_extra: !!python/dict
>>>>> avocado_vt_extra_params:
>>>>> - param1 = 1
>>>>> - param2 = 2
>>>>>
>>>>> stress:
>>>>> test_reference: io-github-autotest-qemu.linux_stress
>>>>>
>>>>> iofuzz:
>>>>> test_reference: io-github-autotest-qemu.iofuzz
>>>>> test_reference_resolver_extra: !!python/dict
>>>>> avocado_vt_extra_params:
>>>>> - param1 = 1
>>>>> - param2 = 2
>>>>>
>>>>> ~$ avocado variants -m testplans/basic.yaml
>>>>> Multiplex variants (4):
>>>>> Variant shutdown-edec: /run/basic/shutdown
>>>>> Variant reboot-c8ba: /run/basic/reboot
>>>>> Variant stress-fd6a: /run/basic/stress
>>>>> Variant iofuzz-e8bc: /run/basic/iofuzz
>>>>> ~$
>>>>>
>>>>> But it is still not accepted by 'avocado {list, run}':
>>>>> ~$ avocado list testplans/basic.yaml
>>>>> Unable to resolve reference(s) 'testplans/basic.yaml' with plugins(s)
>>>>> 'file', 'vt', 'external', try running 'avocado list -V
>>>>> testplans/basic.yaml' to see the details.
>>>>>
>>>>> Do you have any idea why it may happen?
>>>>
>>>> With that single change it works well for me (tried changing the
>>>> "test_reference_resolver_args" and first test's
>>>> "test_reference_resolver_extra" and it successfully propagated to test
>>>> params).
>>>>
>>>
>>>
>>>
>>>> Regards,
>>>> Lukáš
>>>>
>>>>>> autotest==0.16.4
>>>>> avocado-framework==63.0
>>>>> avocado-framework-plugin-varianter-yaml-to-mux==63.0
>>>>> avocado-plugins-vt==63.0
>>>>>
>>>>> Sergey
>>>>>
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Lukáš
>>>>>>
>>>>>> PS: If this happens to work for you, would you please write an
>>>> Avocado-vt
>>>>>> documentation about this? I'd prefer someone external doing so because
>>>>>> (apparently) I take many things as given... (you might as well improve
>>>> the
>>>>>> Avocado documentation, but looking at this I think the Avocado-vt one
>> is
>>>>>> missing more)
>>>>>>
>>>>>>>
>>>>>>> $ avocado run -m basic.yaml
>>>>>>> No test references provided nor any other arguments resolved into
>>>> tests.
>>>>>>> Please double check the executed command.
>>>>>>>
>>>>>>>
>>>>>>> Sergey
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>>>
>>>>>>> чт, 2 авг. 2018 г. в 17:08, Lukáš Doktor <ldoktor at redhat.com>:
>>>>>>>
>>>>>>>> Dne 1.8.2018 v 13:19 Sergey Bronnikov napsal(a):
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> I'm trying to find a way to organize my tests for running without
>>>>>>>>> specifying all of them in command-line. Imagine we have a set of
>>>>>>>> different
>>>>>>>>> tests: sleep.py, passtest.py and fail.py. To run all of them with
>>>>>>>> required
>>>>>>>>> parameters (aka variants) we may execute a command:
>>>>>>>>> avocado run sleep.py fail.py passtest.py -m basic.yaml
>>>>>>>>>
>>>>>>>>> Is there a way to run the same tests without explicit specifying
>>>> them?
>>>>>>>>> For example: avocado run -m basic.yaml
>>>>>>>>>
>>>>>>>>> Sergey
>>>>>>>>>
>>>>>>>>
>>>>>>>> Dear Sergey,
>>>>>>>>
>>>>>>>> as a matter of fact, there are multiple ways.
>>>>>>>>
>>>>>>>> 1. filter-by-tag:
>>>>>>>>
>>>>>>
>>>>
>> http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#categorizing-tests
>>>>>>>>
>>>>>>>> you can tag your tests eg. `sanity`, `release`, ... and then run
>>>>>>>> `avocado run --filter-by-tag=sanity -- TEST_REPO` to only run tests
>>>>>> tagged
>>>>>>>> `sanity` (note the granularity is per-class or even per-test-method.
>>>>>>>>
>>>>>>>> 2. yaml loader optional plugin:
>>>>>>>>
>>>>>>
>>>>
>> http://avocado-framework.readthedocs.io/en/latest/optional_plugins/yaml_loader.html
>>>>>>>>
>>>>>>>> which is probably more what you're looking for, but currently
>> only
>>>>>>>> works on python2 (but should get py3 support, hopefully, soon). It
>>>> works
>>>>>>>> similarly to yaml_to_mux varianter plugin, but allows special keys
>>>> like
>>>>>>>> `test_reference` to specify what test will be used or even
>>>>>>>> `test_reference_resolver_class` when your tests are not the default
>>>>>> ones.
>>>>>>>> This is for example useful when you want to include multiple
>>>>>>>> external-runner tests with different external-runner. There are
>>>> examples
>>>>>>>> in:
>>>>>>>>
>>>>>>
>>>>
>> https://github.com/avocado-framework/avocado/tree/master/examples/yaml_to_mux_loader
>>>>>>>> but let me share my slighly advanced example here:
>>>>>>>>
>>>>>>>> ```yaml
>>>>>>>> # Generated s390x mux-suite
>>>>>>>> !mux
>>>>>>>> qemu_make: !mux
>>>>>>>> mux_suite_test_name_prefix: 'qemu_make/git '
>>>>>>>> check:
>>>>>>>> test_reference:
>>>>>>>> /home/jenkins/s390x/qemu-master/avocado-qemu-make-check
>>>>>>>> install:
>>>>>>>> test_reference:
>>>>>>>> /home/jenkins/s390x/qemu-master/avocado-qemu-make-install
>>>>>>>>
>>>>>>>> unit:
>>>>>>>> variants: !mux
>>>>>>>> test_reference_resolver_class:
>>>>>> avocado.core.loader.ExternalLoader
>>>>>>>> timeout: 360
>>>>>>>> rpm-tcg:
>>>>>>>> mux_suite_test_name_prefix: 'unit/rpm/tcg '
>>>>>>>> test_reference_resolver_extra: !!python/dict
>>>>>>>> loader_options:
>>>>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-tcg
>>>>>>>> rpm-kvm:
>>>>>>>> mux_suite_test_name_prefix: 'unit/rpm/kvm '
>>>>>>>> test_reference_resolver_extra: !!python/dict
>>>>>>>> loader_options:
>>>>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-kvm
>>>>>>>> git-tcg:
>>>>>>>> mux_suite_test_name_prefix: 'unit/git/tcg '
>>>>>>>> test_reference_resolver_extra: !!python/dict
>>>>>>>> loader_options:
>>>>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-tcg
>>>>>>>> git-kvm:
>>>>>>>> mux_suite_test_name_prefix: 'unit/git/kvm '
>>>>>>>> test_reference_resolver_extra: !!python/dict
>>>>>>>> loader_options:
>>>>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-kvm
>>>>>>>> tests: !mux
>>>>>>>> cmm:
>>>>>>>> test_reference: cmm
>>>>>>>> diag10:
>>>>>>>> test_reference: diag10
>>>>>>>> emulator:
>>>>>>>> test_reference: emulator
>>>>>>>> gs:
>>>>>>>> test_reference: gs
>>>>>>>> iep:
>>>>>>>> test_reference: iep
>>>>>>>> intercept:
>>>>>>>> test_reference: intercept
>>>>>>>> pfmf:
>>>>>>>> test_reference: pfmf
>>>>>>>> selftest-setup:
>>>>>>>> test_reference: selftest-setup
>>>>>>>> sieve:
>>>>>>>> test_reference: sieve
>>>>>>>> skey:
>>>>>>>> test_reference: skey
>>>>>>>> sthyi:
>>>>>>>> test_reference: sthyi
>>>>>>>> vector:
>>>>>>>> test_reference: vector
>>>>>>>> functional: !mux
>>>>>>>> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
>>>>>>>> s390x-rpm-libvirt/dvd:
>>>>>>>> mux_suite_test_name_prefix: 'functional/rpm/libvirt/dvd '
>>>>>>>> test_reference:
>>>>>>>> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
>>>>>>>> io-github-autotest-qemu.boot remove_guest.without_disk
>>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_type: libvirt
>>>>>>>> vt_extra_params:
>>>>>>>> - automem=no
>>>>>>>> - url=$URL
>>>>>>>> - vga=none
>>>>>>>> s390x-rpm-libvirt/url:
>>>>>>>> mux_suite_test_name_prefix: 'functional/rpm/libvirt/url '
>>>>>>>> test_reference:
>>>>>>>> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
>>>>>>>> io-github-autotest-qemu.boot remove_guest.without_disk
>>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_type: libvirt
>>>>>>>> vt_extra_params:
>>>>>>>> - automem=no
>>>>>>>> - url=$URL
>>>>>>>> - vga=none
>>>>>>>> s390x-rpm-dvd:
>>>>>>>> mux_suite_test_name_prefix: 'functional/rpm/dvd '
>>>>>>>> test_reference:
>>>>>>>> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
>>>> boot
>>>>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
>>>>>>>> migrate.default.tcp.with_post_copy
>>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_extra_params:
>>>>>>>> - automem=no
>>>>>>>> - url=$URL
>>>>>>>> - ping_pong=5
>>>>>>>> - vga=none
>>>>>>>> s390x-rpm-url:
>>>>>>>> mux_suite_test_name_prefix: 'functional/rpm/url '
>>>>>>>> test_reference:
>>>>>>>> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
>> boot
>>>>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
>>>>>>>> migrate.default.tcp.with_post_copy
>>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
>>>>>>>> vt_extra_params:
>>>>>>>> - automem=no
>>>>>>>> - url=$URL
>>>>>>>> - ping_pong=5
>>>>>>>> - vga=none
>>>>>>>> s390x-git-dvd:
>>>>>>>> mux_suite_test_name_prefix: 'functional/git/dvd '
>>>>>>>> test_reference:
>>>>>>>> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
>>>> boot
>>>>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
>>>>>>>> migrate.default.tcp.with_post_copy
>>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>>> vt_qemu_bin:
>>>>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
>>>>>>>> vt_dst_qemu_bin:
>>>>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
>>>>>>>> vt_extra_params:
>>>>>>>> - automem=no
>>>>>>>> - url=$URL
>>>>>>>> - ping_pong=5
>>>>>>>> - vga=none
>>>>>>>> s390x-git-url:
>>>>>>>> mux_suite_test_name_prefix: 'functional/git/url '
>>>>>>>> test_reference:
>>>>>>>> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
>> boot
>>>>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
>>>>>>>> migrate.default.tcp.with_post_copy
>>>>>>>> test_reference_resolver_args: !!python/dict
>>>>>>>> vt_qemu_bin:
>>>>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
>>>>>>>> vt_dst_qemu_bin:
>>>>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
>>>>>>>> vt_extra_params:
>>>>>>>> - automem=no
>>>>>>>> - url=$URL
>>>>>>>> - ping_pong=5
>>>>>>>> - vga=none
>>>>>>>> ```
>>>>>>>>
>>>>>>>> (feel free to ask for details, but basically it uses SIMPLE tests,
>>>>>>>> EXTERNAL_RUNNER tests and then Avocado-vt tests. The test always
>>>>>> receives
>>>>>>>> all the params so you can override the standard arguments. If you
>>>>>> specify
>>>>>>>> "test_reference_resolver_args" it also updates the loader
>> parameters,
>>>>>> which
>>>>>>>> is what I use for Avocado-vt tests).
>>>>>>>>
>>>>>>>> Theoretically there is yet another way and that is to create
>> directory
>>>>>>>> "sanity" and put symlinks to actual tests there, but the granularity
>>>>>> there
>>>>>>>> would be only per-test-class, which usually does not suffice.
>>>>>>>>
>>>>>>>> Anyway, hopefully at least one method suits you, but feel free to
>>>>>>>> elaborate more, we can always improve.
>>>>>>>>
>>>>>>>> Kind regards,
>>>>>>>> Lukáš
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20180808/6eb73d29/attachment.sig>
More information about the Avocado-devel
mailing list