[Freeipa-devel] Releasing testing tools as standalone projects

[Petr Viktorin]

On 11/06/2014 05:00 PM, Scott Poore wrote:
> ----- Original Message -----
>> From: "Petr Viktorin" <pviktori at redhat.com>
[...]
>> I've opened a ticket to get the project space for the BeakerLib plugin:
>> https://fedorahosted.org/fedora-infrastructure/ticket/4589
>> When that's done I'll point the package metadata to there, push to PyPI
>> and open a Fedora review request.
>
> Awesome.  Thanks, Petr.
>
>>
>> While we wait, it's available here:
>> https://github.com/encukou/pytest-beakerlib
>> https://copr.fedoraproject.org/coprs/pviktori/pytest-beakerlib
>>
>
> Mind if I pass this on to some other QE teams that might be interested?

Not at all, go ahead!
You might want to wait until the project gets its permanent home, though.

>
>> Do we need a version for EL 6? I'd need to check the pytest versions
>> there, and build a newer pytest if necessary.
>
> Yes, I think we will want an EL6 version as well at least at some point.

Let me know when that point comes :)

>> The second part is the multi-host framework. I've looked at what parts
>> are applicable to other projects than IPA, and came up with an initial
>> design/README here: https://github.com/encukou/pytest-multihost
>> I'll add a concrete example, code, and patches for IPA, soon.
>
> This is the paramiko/openssh stuff you've mentioned before right?
>
> I think this is the other piece I'd be very interested in.

Yup, that's what it is.

>> IPA also has/will have a plugin to run tests within a class in source
>> order (respecting inheritance), rather than in pytest's unspecified
>> order ("usually alphabetically", IIRC). It can be extracted as well if
>> there's interest.
>>
>
> When you say in source order here, you mean source code order?  So, we could actually order tests in the file as we see fit instead of relying on naming to define execution order?

Yes. (Except it orders per class, not per source file.)

It's not good practice to rely on test order, but it works for 
integration tests with big setup costs.

> Would this affect use of the built in setup/teardown fixtures?  Or we should just stay away from those anyway?

I'm not sure what you mean by built in setup/teardown fixtures?
Anyway it should be safe, unless you use another test reordering plugin 
as well. (e.g. there's a 3rd party plugin for distributed running of 
tests, using that would not be a good idea)


-- 
Petr³