[Avocado-devel] Running tests in parallel

[Cleber Rosa]

On 11/23/2016 07:07 AM, Zubair Lutfullah Kakakhel wrote:
> Hi,
> 
> Thank you for your comprehensive reply!
> 
> Comments inline.
> 
> On 11/22/2016 02:11 PM, Cleber Rosa wrote:
>> On 11/22/2016 07:53 AM, Zubair Lutfullah Kakakhel wrote:
>>> Hi,
>>>
>>
>> Hi Zubair,
>>
>>> There are quite a few threads about this and a trello card
>>> https://trello.com/c/xNeR2slj/255-support-running-tests-in-parallel
>>>
>>> And the discussion leads to a complex multi-host RFC.
>>> https://www.redhat.com/archives/avocado-devel/2016-March/msg00025.html
>>>
>>> Our requirement is simpler.
>>> All we wanted to do is run disjoint simple (c executables) tests in
>>> parallel.
>>>
>>
>> Sounds fair enough.
>>
>>> I was wondering if somebody has a WIP branch that has some level of
>>> implementation for this?
>>
>> I'm not familiar with a WiP or PoC on this (yet).  If anyone has
>> experimented with it, I'd happy to hear about it.
>>
>>> Or If somebody is familiar with the code base, I'd appreciate some
>>> direction on how to implement this.
>>>
>>
>> Avocado already runs every single test in a fresh new process.  This is,
>> at least theoretically,  a good start.  Also, the test process is
>> handled based on the standard Python multiprocessing module:
>>
>> https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L363
>>
>>
>> The first experimentation I'd do would be to attempt using the also
>> Python standard multiprocessing.Pool:
>>
>> https://docs.python.org/2.7/library/multiprocessing.html#using-a-pool-of-workers
>>
> 
> In this case, there would be a separate python thread for each test
> being run in parallel.
> Each python thread would actually call the test executable using a
> sub-process?
> 

Ideally, the Avocado test runner would remain a single process, that is,
without one additional thread (or process) to manage each *test* process.

> That can be OK for Desktops but won't scale well for using avocado in
> memory
> constrained Embedded devices.
> 

I must admit I haven't attempted to run Avocado in resource constrained
environments.  Can you explain what is your bigger concern?

Do you feel that Avocado (as a single process test *runner*) plus one
process for each *test* is not suitable to those environments?

- Cleber.

> Please correct me if I am reading this incorrectly.
> 
> Regards,
> ZubairLK
> 
>>
>> This would most certainly lead to changes in how Avocado currently
>> serially waits for the test status:
>>
>> https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L403
>>
>>
>> Which ultimately is added to the (Job wide) results:
>>
>> https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L455
>>
>>
>> Since the results for many tests will now be acquired in unpredictable
>> order, this will require changes to the ResultEvent based plugins (such
>> as the UI).
>>
>>> Thanks
>>>
>>> Regards,
>>> ZubairLK
>>>
>>
>> I hope this is a good initial set of pointers.  If you feel adventurous
>> and wants to start hacking on this, you're more then welcome.
>>
>> BTW: we've had quite a number of features that started as
>> experiments/ideas/not-really-perfect-pull-requests from the community
>> that Avocado "core team" members embraced and pushed all the way to
>> completeness.
>>

-- 
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20161123/924e7931/attachment.sig>