[Avocado-devel] Fwd: Avocado future

[Lukáš Doktor]

Dne 04. 12. 18 v 16:40 Laurent Leuvrey napsal(a):
> Hello,
> 

Hello Laurent,

I'm sorry for such a late response, I wanted to double-check with the core-team first before answering. First heads-up would be, internally we meet on Monday and every release there is a public meeting where you can get immediate response (and get it recorded, see https://www.youtube.com/channel/UC-RVZ_HFTbEztDM7wNY4NfA Another chance could be the avocado+qemu meeting every Tuesday https://www.redhat.com/archives/avocado-devel/2018-November/msg00018.html that is dedicated to avocado_qemu work, but I believe people will talk to you about regular Avocado issues as well.

>     Trying to gather information on what would be Avocado in the future, I initially sent the below mail to the developpers mailing list as it was the only way of contacting Avocado team I found in the doc. Then the email has been returned. That's why I would take the liberty to directly reach you.
> 
>     Best regards,
> 
>             Laurent
> 
> -------- Message transféré --------
> Sujet :         Avocado future
> Date :  Tue, 4 Dec 2018 16:15:12 +0100
> De :    Laurent Leuvrey <laurent.leuvrey at arm.com><mailto:laurent.leuvrey at arm.com>
> Pour :  avocado-devel at redhat.com<mailto:avocado-devel at redhat.com>
> 

This is odd, don't know why it rejected you. Maybe try subscribing there first.

> 
> Hello,
> 
>     As I'm looking for a testing framework suite and Avocado is very appealing regarding the functional test, its active development and quickly evolving features. Then I went through documentation, forum and video to know what will be Avocado's future, if its testing abilities will grow. And I still got some questions regarding Avocado's evolution :

We definitely plan on improving Avocado, RFEs are warmly welcome as well as contributions. Lately we focused more on avocado+qemu, but we have other teams at RH using it for userspace/kernel testing and there are other companies using it for various testing as well.

> 
>     - is there a plan to support unit testing features through plugins for tools such as xUnit tools (JUnit, CppUnit...)
> 

Integrating with different unit-testing tools should be fairly simple, we have glib/golang/python-unittest/robot loader plugins so adding new ones should be doable.

>     - will Avocado be able to use distributed computing (LSF, Cloud...) for parallel testing jobs submission for instance
> 

I'm not aware of any plans to do distributed parallel testing as we usually use Avocado as the "man-in-the-middle" and Jenkins as the tool to decide where and how to run. Anyway you can split jobs and gather results as a collection of xunit/tap, but sure, a way to balance this would be nice. Again, RFE is a nice way to get feedback. (I know there is already EC2 plugin and somewhere I saw Azure plugin as well, but don't have real experience with these)

>     - currently, there are some wrappers for tools such as Valgrind. It allows to instrument a test run. Will it be possible, thanks to Avocado command line options, to ask Avocado to directly instrument the tested command itself. For instance, you want to test the "sleep" command, then you write a sleepTest.py "around" this command. For now Valgrind will estimate the sleepTest.py memory usage but not the system sleep itself.
> 

I like these as well as providing detailed feedback on failed tests is as simple as re-running it with certain tools enabled. Not sure what you really want but I think wrappers work the way you want. When you write a test, whenever you use `avocado.utils.process` library to execute anything, that is the point where you can hook your wrapper/gdb. The simplest demonstration would be:

    avocado --show all run --external-runner /bin/sleep --wrapper "/bin/time:*sleep" -- 1

Which won't run the whole "ExternalRunnerTest" via `/bin/time`, but it will execute the ExternalRunnerTest, which uses `process.run` to execute the desired binary. Now the `--wrapper $script:binary` makes sure whenever `process.run` attempts to run matching binary ("*sleep" in this example) it uses the wrapper.

Now when you create a complex test with many `process.run` invocations, each of them will check for the binary and on match uses that wrapper/gdb. Btw gdb is even cooler than wrappers as it allows normal execution, reports stdout/stderr and everything and only on breakpoint you get a nice UI message how to connect to the gdb server. Very useful for thins that require many iterations to reproduce.

Best regards,
Lukáš

>     Best regards,
> 
>                 Laurent
> 
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20181211/bde3f0e5/attachment.sig>