[Avocado-devel] RFC: Test Developers API

[Lucas Meneghel Rodrigues]

On Fri, Mar 27, 2015 at 12:39 PM, Ruda Moura <rmoura at redhat.com> wrote:
> Test Developers API
> ===================
> 
> From the discussion we've been doing so far, there are four
> major ideas emerging to give the test developers
> a clear instruction on what is the API for creating
> tests in Avocado.

Here comes my review and vote: Spoiler alert, I vote in proposal 4.

If you want to see my reasoning, read on.

> 
> 
> https://trello.com/c/ENlP9vAu/351-study-review-test-api-set-a-boundary-for-test-writers
> 
> 
> 1. Export Test developers API in avocado main module
> ----------------------------------------------------
> 
> Just export the classes, methods and functions
> inside the avocado.__init__ module.
> 
>   from avocado import
>   avocado.Test, avocado.run, avocado.build, etc.
> 
> If we defined __all__ = [Test, run, system, etc.]
> then we can even control what the users will
> get if he/she does 'from avocado import *'.
> 
> Unittest and nose does it.

Unittest and nose also have a more limited scope in what's API on their 
terms. they don't have replacements for subprocess and code that is 
useful for testing, they only have the base unittest class and that's 
basically it. Avocado works as both a provider of applications (think 
nose) and libraries (think django), so it is a bit different. This is 
similar to the .api namespace, except that every useful symbol is 
exported at the root module, forcing the usable API to be disposed on a 
flat namespace. I don't like this option.

> 
> 
> 2. Export Test developers API in submodule avocado.api
> ------------------------------------------------------
> 
> In this case, avocado.api or avocado.testing or any better name,
> contains the whole API for test developers.
> The point here is to isolate the test API in a submodule.
> 
> It has a clear border for test developers.
> So it will contain Test classes and methods,
> functions to run process, logging etc.
> 
> The Fabric project does it (see fabric.api).

That seems like a passable solution, it's a clear boundary, although it 
was pointed out that the 'natural' reaction of users inspecting the 
module is to assume that avocado is the base namespace exposed for them 
to use, so it's probably better to do the reverse (see proposal 4). 
Another point, already discussed in proposal 1 is that this naturally 
leads to a flat namespace, where we have in the same level functions to 
run subprocess, but also functions to uncompress files, functions to 
list distro packages, to build tarballs and so on and so forth. I don't 
like this solution.

> 
> 
> 3. Using a different namespace than avocado
> -------------------------------------------
> 
> We just create a different namespace (like avocado_core),
> instead of using everything under avocado.
> We will keep the test API under a new
> namespace avocado_core or avocado_framework.
> 
> The drawback here is how to manage separated projects.

IMO, the drawback is that having avocadocore and avocado (or any other 
naming convention, really) does not strike me as an elegant solution. 
We could have both namespaces inside the same project with minimal 
hassle. I don't like this solution.

> 
> 
> 4. Test Developers: "Don't use avocado.app"
> -------------------------------------------
> 
> This is a different approach than the others above.
> In this case, we concentrate everything that isn't
> for test developers under a new submodule avocado.app.
> 
> Everything related to plugins, parsing of command line,
> logging, and any stuff target to the application, goes to avocado.app.
> So the test developers can use everything else from avocado.
> 
> The potential problem here is cyclical dependences.

I don't see a lot of potential cyclical dependencies. It is fine for 
avocado.app to depend on the rest of avocado, but it's not fine the 
rest of avocado to depend on avocado.app. That should be simple enough, 
although it is easier said than done.

That's my favorite option so far, since avocado.app should be a 
distinctive namespace, self explanatory and reduces the confusion (I'm 
hard pressed to come up with a scenario that people think it's OK to 
use avocado.app imports in their tests. Moreover, documentation exists 
to say 'don't use this in your tests').