[Avocado-devel] How to execute different tests of avocado-vt in one round
Vit Pelcak
vpelcak at suse.cz
Thu Jan 10 08:09:21 UTC 2019
Hello.
Thank you very much for all the info.
Really appreciated.
Best regards
Vit Pelcak
Dne 09. 01. 19 v 18:23 Lukáš Doktor napsal(a):
> Dne 09. 01. 19 v 15:57 Vit Pelcak napsal(a):
>> Hello.
>>
>> Dne 08. 01. 19 v 17:47 Cleber Rosa napsal(a):
>>>
>>>
>>> On 1/8/19 9:20 AM, Brian J. Murrell wrote:
>>>> On Tue, 2019-01-08 at 14:57 +0100, Vit Pelcak wrote:
>>>>> Hello.
>>>>
>>>> Hi,
>>>>
>>>> While not really answering your question, per se but perhaps you are
>>>> trying to solve the wrong problem...
>>>>
>>>>>
>>>>> That will mean extra 10 minutes for each of them as
>>>>> VM
>>>>> will have to be started each time, resulting in extra 30 minutes time
>>>>> in
>>>>> total.
>>>>
>>>> Your virtual machine really takes 10 minutes to start? Why so long?
>>>> It should be a minute or two at most. Under a minute would not be
>>>> absurd.
>>>>
>>>> Cheers,
>>>> b.
>>>>
>>>
>>> Unless you're running under CPU emulation, or nested virt, low powered
>>> and highly loaded environments...
>>>
>>> Still, based on experience, we strongly suggest you start a fresh VM on
>>> every new test: the results can be easily spoiled if you don't do that.
>>
>> Thank you very much for advice.
>>
>> And if I do this:
>>
>>
>> https://github.com/avocado-framework/avocado-vt/blob/master/etc/avocado/conf.d/vt.conf#L7
>>
>> # Keep guest running between tests (faster, but unsafe)
>> keep_guest_running=False
>>
>> Is it possible to trigger VM restart?
>>
>
> As mentioned in my other email, VM gets recreated when test requires different devices/setting. You can also execute "shutdown" test. Last but the most powerful would be the yaml_to_mux loader that allows you to fine-tune your setting. Excerpt from my CI:
>
> ```
> # Generated s390x mux-suite
> !mux
> qemu_make: !mux
> mux_suite_test_name_prefix: 'qemu_make/git '
> check:
> test_reference: /home/jenkins/s390x/qemu-master/avocado-qemu-make-check
> install:
> test_reference: /home/jenkins/s390x/qemu-master/avocado-qemu-make-install
>
> unit:
> variants: !mux
> test_reference_resolver_class: avocado.core.loader.ExternalLoader
> timeout: 360
> rpm-tcg:
> mux_suite_test_name_prefix: 'unit/rpm/tcg '
> test_reference_resolver_extra: !!python/dict
> loader_options: /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-tcg
> rpm-kvm:
> mux_suite_test_name_prefix: 'unit/rpm/kvm '
> test_reference_resolver_extra: !!python/dict
> loader_options: /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-kvm
> # ...
> tests: !mux
> cmm:
> test_reference: cmm
> diag10:
> test_reference: diag10
> emulator:
> test_reference: emulator
> # ...
>
> functional: !mux
> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
> s390x-rpm-libvirt/dvdfetch_dvd/:
> mux_suite_test_name_prefix: 'functional/rpm/libvirt/dvd '
> test_reference_resolver_class: avocado.core.loader.ExternalLoader
> test_reference_resolver_extra: !!python/dict
> loader_options: /bin/sh -c
> test_reference: "'avocado_get_rhel7_cd s390x \"URL\" '"
> s390x-rpm-libvirt/dvd:
> mux_suite_test_name_prefix: 'functional/rpm/libvirt/dvd '
> test_reference: unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads io-github-autotest-qemu.boot remove_guest.without_disk
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_type: libvirt
> vt_extra_params:
> - automem=no
> - qemu_sandbox=''
> - url=URL
> - vga=none
> - mem=2048
> s390x-rpm-libvirt/url:
> mux_suite_test_name_prefix: 'functional/rpm/libvirt/url '
> test_reference: unattended_install.url.extra_cdrom_ks.default_install.aio_threads io-github-autotest-qemu.boot remove_guest.without_disk
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_type: libvirt
> vt_extra_params:
> - automem=no
> - qemu_sandbox=''
> - url=URL
> - vga=none
> - mem=2048
> s390x-rpm-RHEL-7/install/dvd/fetch_dvd/:
> mux_suite_test_name_prefix: 'functional/rpm/RHEL-7/install/dvd/ '
> test_reference_resolver_class: avocado.core.loader.ExternalLoader
> test_reference_resolver_extra: !!python/dict
> loader_options: /bin/sh -c
> test_reference: "'avocado_get_rhel7_cd s390x \"URL\" '"
> s390x-rpm-RHEL-7/install/dvd/:
> mux_suite_test_name_prefix: 'functional/rpm/RHEL-7/install/dvd/ '
> test_reference: unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
> test_reference_resolver_args: !!python/dict
> vt_qemu_bin: /usr/libexec/qemu-kvm
> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> vt_extra_params:
> - automem=no
> - qemu_sandbox=''
> - url=URL
> - ping_pong=5
> - vga=none
> - mem=2048
> - timeout=3600
> # ...
>
> ```
>
> Which is simply stored in "suite.yaml" file and executed by "avocado run suite.yaml". It executes few SIMPLE tests, then few EXTERNAL_RUNNER tests and then some VT tests with different --vt... options. (the "# ..." means I skipped some boring parts; URLs were also removed...)
>
> Regards,
> Lukáš
>
>> I am thinking that it could make sense to group tests, disable VM
>> restart and then restart the machine after each group of tests is finished.
>>
>>> Regards!
>>>
>>
>
>
--
Vit Pelcak vpelcak at suse.cz
Team Lead in QA/Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20190110/503e7042/attachment.sig>
More information about the Avocado-devel
mailing list