[Avocado-devel] Is it possible to handle variants only for some tests from cli interface?

[Lukáš Doktor]

I'm away from the project for some time, but what is the status of the "varianter_yaml_to_mux"? It used to be designed for that and the way it worked was you created one big yaml file and created all kinds of combinations. I used to use it to combine various kinds of jobs with different options and even multiple heterogeneous variants.

Regards,
Lukáš

Dne 19. 06. 23 v 9:58 Jan Richter napsal(a):
> On 6/16/23 12:33, Uladzimir Bely wrote:
>> Hello all.
>>
>> I have some "tests.py" with two test classes I called MuxTest and NoMuxTest.
>>
>> I want to run tests from MuxTest several times with different options, while tests from NoMuxTest should be run only once (they don't have any parameters for change).
>>
>> I'm able to do it with python avocado interface using Job/TestSuite:
>>
>> dict_variants.py:
>>
>> ```
>> import sys
>>
>> from avocado.core.job import Job  
>> from avocado.core.suite import TestSuite
>>
>> MUX_CONFIG = {  
>> "resolver.references": ["tests.py:MuxTest"],  
>> "run.dict_variants.variant_id_keys": ["dut"],  
>> "run.dict_variants": [  
>> {"dut": "rpi-arm-v7l"},  
>> {"dut": "de0-nano-soc"},  
>> {"dut": "stm32mp15x"}  
>> ],  
>> }
>>
>> NOMUX_CONFIG = {  
>> "resolver.references": ["tests.py:NoMuxTest"],  
>> }
>>
>> with Job(  
>> test_suites=[  
>> TestSuite.from_config(MUX_CONFIG, name="mux"),  
>> TestSuite.from_config(NOMUX_CONFIG, name="nomux"),  
>> ]  
>> ) as j:  
>> sys.exit(j.run())
>> ```
>>
>> It works like expected. When running dict_variants.py I get:
>>
>> ```
>> $ ./dict_variants.py 
>> JOB ID     : 37bc94579fe96aad053439615ce4f76ffb91406a
>> JOB LOG    : /home/wiselord/avocado/job-results/job-2023-06-16T13.21-37bc945/job.log
>>  (mtda-1/3) mtda.py:MuxTest.test_hw;rpi-arm-v7l: STARTED
>>  (mtda-1/3) mtda.py:MuxTest.test_hw;rpi-arm-v7l: PASS (0.01 s)
>>  (mtda-2/3) mtda.py:MuxTest.test_hw;de0-nano-soc: STARTED
>>  (mtda-2/3) mtda.py:MuxTest.test_hw;de0-nano-soc: PASS (0.01 s)
>>  (mtda-3/3) mtda.py:MuxTest.test_hw;stm32mp15x: STARTED
>>  (mtda-3/3) mtda.py:MuxTest.test_hw;stm32mp15x: PASS (0.01 s)
>>  (nomux-1/1) mtda.py:NoMuxTest.test_nomux: STARTED
>>  (nomux-1/1) mtda.py:NoMuxTest.test_nomux: PASS (0.01 s)
>> RESULTS    : PASS 4 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME   : 1.98 s
>> ```
>>
>> mtda.py:MuxTest.test_hw is run 3 times with different parameters, while mtda.py:NoMuxTest.test_nomux is run only once.
>>
>> My question is the following:
>>
>> - Is it possible to achieve the same behaviour using avocado's command line interface (e.g. "avocado run mtda.py ...")?
>>
>> I tried to experiment, but either variants applied for both tests (so, I have 6 runs totally), or to none (so I have 2 runs).
>>
> 
> Hi Uladzimir,
> 
> avocado's command line interface creates only one Job with one TestSuite
> and right now avocado doesn't support multiple TestSuites through the
> command line. Therefore, you can't achieve the same results with command
> line interface. IMO, your approach with Job API is correct. If you
> really need to use command line interface, you have to use two different
> avocado run commands, but your tests will be in two different avocado jobs.
> 
> — Jan
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x26B362E47FCF22C1.asc
Type: application/pgp-keys
Size: 12726 bytes
Desc: OpenPGP public key
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20230622/c85d755d/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20230622/c85d755d/attachment.sig>