[Avocado-devel] How to execute different tests of avocado-vt in one round

[Lukáš Doktor]

Dne 08. 01. 19 v 17:43 Cleber Rosa napsal(a):
> 
> 
> On 1/8/19 8:57 AM, Vit Pelcak wrote:
>> Hello.
>>
>> I would like to ask for help how to run several different tests in
>> single run so that virtual machine doesn't have to be started for every
>> single one of them.
>>
>> For example running:
>> type_specific.io-github-autotest-qemu.multi_nics_hotplug
>> type_specific.io-github-autotest-qemu.qemu_disk_img.info.backing_chain
>> type_specific.io-github-autotest-qemu.qemu_io_blkdebug
>> type_specific.io-github-autotest-qemu.qmp_command
>>
>> will mean that I will have to run "avocado run --vt-type qemu xyz" for
>> each one of them. That will mean extra 10 minutes for each of them as VM
>> will have to be started each time, resulting in extra 30 minutes time in
>> total.
>>
>> Is it possible to overcome this so that virtual machine is started only
>> once and all 4 tests are run on it? And I do not want to run all tests
>> in "type_specific.io-github-autotest-qemu", of course.
>>
> 
> Should be doable by setting this to true:
> 
> https://github.com/avocado-framework/avocado-vt/blob/master/etc/avocado/conf.d/vt.conf#L7
> 
> # Keep guest running between tests (faster, but unsafe)
> keep_guest_running=False
> 

Yes, this is the key but also you should disable "restore_image_after_testing" to avoid overriding image that is still in use. Again, weird things would happen. We do workaround it by first deleting the image, so it'd work well, but when you try migration it'd still use the deleted image on the source side, but the destination image would get the refreshed image. Another caveat is that after failed test you could only get the actual image by looking at deleted fds (ideally from qemu opened fds) because the image you see is not the image that the VM is using. It also happened that when the test required different devices/ports/... it was not detected and it received the previous VM. It was especially tricky for spice. It is solved now, but noone knows when another issue pops up.

Anyway setting this does not mean VMs will be kept alive. When the next test requires different devices it will get (hopefully) recreated (for example multi_nics_hotplug will probably end-up with some extra devices...).


As nowadays starting VMs takes less and less (usually under 20s) I prefer clean environment so I can't tell whether this option still works as expected.

Speaking of performance, the "restore_image_after_testing" is actually the one I'd advocate for. You can set Avocado to refresh image on failure (eg. by `--vt-extra-params restore_image_after_testing_on_error=yes`) while keeping it in case of passed test (by setting restore_image_after_testing=no in vt.conf, or directly by --vt-extra-params). Yes, previous test(s) would be able to spoil your environment, but you'll gain some significant time. Refreshing images takes usually longer than starting a VM (not for JeOS, but for example RHEL.7.devel is 5.5G so without SSD it takes a while).

Also note that you can use "yaml2mux loader" to fine-tune those options and create groups of tests that will keep vm alive, then let it refresh and then create another group where you don't refresh the image after testing. But for me the "on_error" handling is usually enough.

Regards,
Lukáš


> The net effect is that 'kill_vm*' parameters to the tests are modified.
> 
> Regards,
> - Cleber.
> 
>> Thank you very much.
>> Best regards
>>
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20190109/8857ed54/attachment.sig>