[Avocado-devel] Tests stable tmpdir

[Cleber Rosa]

On 10/30/2016 06:37 AM, Fam Zheng wrote:
> On Thu, 10/27 15:28, Ademar Reis wrote:
>>>
>>> That indeed becomes true when we start offering the locking mechanism.
>>> Right now, our users simply want/need to do setup that is valid for many
>>> (usually all) tests.  According to Amador, the lack of a such a
>>> mechanism, has led users to write larger tests, when they should really
>>> be smaller ones.
>>
>> So we can offer the contrib plugin without the locking mechanism,
>> leaving it to users to decide what to do with it. Documented as a
>> non-supported feature.
>>
>> As we learn more about this use-case, we can expose it as a
>> fully-supported API.
> 
> Just want to add two cents on how sharing resources across multiple tests can be
> useful:
> 
> Sometimes a test itself is way quicker than the preparation, and the latter on
> the other hand could ususally be common across many tests. Without a mechanism
> to reuse a setup across multiple tests cases (at least inside the same test
> class), test cases are forced to be combined into one. That is, when a test
> script would ideally look like this:

It's true that test setups can be many many times more expensive than
the test themselves.  We've seen many "solutions" to that, including
considering a large setup *phase* a test itself.  Then we'd have other
tests that depend on this "setup test".

What all examples have in common is that there must be knowledge about
some common state.  And besides knowledge, sometimes real assets.

What Avocado has tried to set as a basic principle, is that tests should
not depend on "other tests", "job state", "shared resources", etc.  I
believe this is a good thing because it keeps the architecture clean and
can allow advanced use cases (most of which we haven't implemented yet).

> 
>     class MyTestCase:
>         def setUp(self):
>             self.do_a_complicated_setup()
> 
>         def test1():
>             self.a_quick_test();
> 
>         def test2():
>             self.another_quick_test();
> 
>         def test3():
>             self.yet_another_quick_test();
> 

But, we cannot deny that this example is better than the second one.
One approach, as mentioned earlier, is to move the commonalities to the
job phase.  Still, we have said that tests shouldn't have knowledge
about their job.  Also, we can certainly see users needing jobs in which
different sets of tests need different sets of common setups.

So, what I see, and this is a brainstorm, is that Avocado will need in
the future, a way for users to opt out of this complete test execution
independence.  Please don't be alarmed at this point, I'm not
suggesting breaking our core foundations.

What I'm suggesting is a way for users to note that a set of tests are
related, and they should share more, such as the same machine or
temporary dir.  A command line switch such as
`--keep-together=<none|class|all>` could do the trick.

> It would be squashed into one test case like this:
> 
>     class MyTestCase:
>         def setUp(self):
>             self.do_a_complicated_setup()
> 
>         def testAll():
>             self.a_quick_test();
>             self.another_quick_test();
>             self.yet_another_quick_test();
> 
> 
> I think the first way is much cleaner because the function names can be
> self-explanatory.

Agreed.  This is what Amador mentioned that ends up getting done.  You
miss the right granularity, and if a regression is introduced, it's not
crystal clear what actually broke.

> 
> Fam
> 

-- 
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20161031/4d70406f/attachment.sig>