[Avocado-devel] Tests stable tmpdir

[Lukáš Doktor]

Dne 25.10.2016 v 17:15 Ademar Reis napsal(a):
> On Mon, Oct 24, 2016 at 06:14:14PM -0300, Cleber Rosa wrote:
>>
>> On 10/24/2016 10:27 AM, Amador Pahim wrote:
>>> Hello,
>>>
>>> I saw a number of requests about setUpClass/tearDownClass. We don't
>>> actually support them in Avocado, as already stated in our docs, but
>>> most of the requests are actually interested in have a temporary
>>> directory that can be the same throughout the job, so every test can
>>> use that directory to share information that is common to all the
>>> tests.
>>>
>>> One way to provide that would be exposing the Job temporary directory,
>>> but providing a supported API where a test can actually write to
>>> another test results can break our promise that tests are independent
>>> from each other.
>>>
>>
>> Yes, the initial goal of a job temporary directory is to prevent clashes
>> and allow proper cleanup when a job is finished.  For those not familiar
>> with the current problems of (global) temporary directories:
>>
>> https://trello.com/c/qgSTIK0Y/859-single-data-dir-get-tmp-dir-per-interpreter-breaks-multiple-jobs
>
> Also, let's keep in mind that the architecture of Avocado is
> hierarchical and tests should not have access or knowledge about
> the job they're running on (I honestly don't know how much of
> this is true in practice today, but if it happens somewhere, it
> should be considered a problem).
>
> Anyway, what I want to say is that we should not expose a job
> directory to tests.
>
>>
>>
>>> Another way that comes to my mind is to use the pre/post plugin to
>>> handle that. On `pre`, we can create a temporary directory and set an
>>> environment variable with the path for it. On `post` we remove that
>>> directory. Something like:
>>>
>>> ```
>>> class TestsTmpdir(JobPre, JobPost):
>>>     ...
>>>
>>>     def pre(self, job):
>>>         os.environ['AVOCADO_TESTS_TMPDIR'] = tempfile.mkdtemp(prefix='avocado_')
>>>
>>>     def post(self, job):
>>>         if os.environ.get('AVOCADO_TESTS_TMPDIR') is not None:
>>>             shutil.rmtree(os.environ.get('AVOCADO_TESTS_TMPDIR'))
>>> ```
>>>
>>> Thoughts?
>>>
>>
Honestly I'd prefer this to be part of the API. Something like 
`Test.shared_job_tmp` or so. But I know tests are not suppose to be 
shared, it's not right, etc. but in reality they always are (unless we 
create a new clean machine/container for each test).

Given that this solution is the closest to what I'd propose. I'd be 
honest so I'd call it `SharedJobTmpdir` and `AVOCADO_SHARED_JOB_TMP`, 
because that is what it is (no matter how we want to avoid the job, but 
it creates the dir during pre-job and removes it during post-job).

Btw to give another alternative, one can always use `--mux-inject` or 
env variables to add shared dir manually. But I consider this cumbersome.

>> I think this can be a valid solution, that promises very little to
>> tests.  It doesn't break our assumption of how tests should not depend
>> on each other, and it reinforces that we aim at providing job level
>> orchestration.
>
> Thinking from the architecture perspective once again, this is a
> bit different from what you proposed before, but not that much
> (let's say it's a third-party "entity" called
> "AVOCADO_TESTS_TMPDIR" available to all processes in the job
> environment, unique per job).
>
> It's a bit better, but first of all, it should be named,
> implemented and even enabled in a more explicit way to prevent
> users from abusing it.
>
> But my real solution is below:
>
>>
>> Although, since we have discussed giving a job its own temporary dir,
>> and we already expose a lot via environment variables to tests:
>>
>> http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#environment-variables-for-simple-tests
>>
>> And also to job pre/post script plugins:
>>
>> http://avocado-framework.readthedocs.io/en/latest/ReferenceGuide.html#script-execution-environment
>>
>> I'm afraid this could bring inconsistencies or clashes in the very near
>> future.  What I propose for the immediate terms is to write a
>> contrib/example plugin, that we can either fold into the Job class
>> itself (giving it a real temporary dir, with variables exposed to test
>> processes) or make it a 1st class plugin.
>>
>> How does it sound?
>
> If we expose something like this as a supported API, we should
> make it as an "external resource available for tests" with proper
> access control (locking) mechanisms. In other words, this feature
> is a lot more about the locking API than about a global directory
> for tests.
>
> In summary, a "job/global directory available to all tests"
> should in fact be handled as "a global resource available to all
> tests".  Notice it has no relationship to jobs whatsoever.
> Creating it per-job would be simply an implementation detail.
>
> Think of the hypothetical examples below and consider the
> architectural implication:
>
> (all tests in these examples are making use of the shared dir)
>
> $ export MY_DIR=~/tmp/foobar
> $ avocado run my-test.py
> $ avocado run my-test1.py my-test2.py
> $ avocado run my-test.py & avocado run my-test.py
> $ avocado run --enable-parallel-run my-test*.py
>
> Some of the above will break with today's Avocado. Now imagine we
> provide a locking API for shared resources. Tests could then do
> this:
>
>     lock($MY_DIR)
>       do_something...
>     unlock($MY_DIR)
>
> Or maybe even better, simply declare they're using $MY_DIR during
> their entire execution via a decorator or some (future)
> dependency API:
>
>     @using($MY_DIR)
>     def ...
>
> With that in place, we could have a plugin, or even a first-class
> citizen API, to create and expose a unique directory per job.
>
> Thanks.
>    - Ademar
>

There are several existing libraries to lock directories/files. I don't 
think we need to implement them as one can download them from pip. The 
only thing user require is the shared dir and than they can do:

```
from lockfile import LockFile

def setUp(self):
     self.shared_dir = os.get("AVOCADO_SHARED_JOB_TMP")
     assert self.shared_dir
     with LockFile(self.shared_dir):
         # do my stuff
     with LockFile(os.path.join(self.shared_dir, "qemu")):
         # do something inside qemu while allowing other tests
         # to access shared_dir
```

Yes, we can eventually create our version and promote it in our 
`avocado.utils` but even without it I see a big benefit in having share dir.

Lukáš

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 502 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20161031/b03a8e79/attachment.sig>