[Freeipa-devel] [PATCH] Workaround for trac N 5348

[Milan Kubík]

On 10/08/2015 02:50 PM, Martin Basti wrote:
>
>
> On 10/08/2015 02:39 PM, Martin Kosek wrote:
>> On 10/08/2015 02:08 PM, Oleg Fayans wrote:
>>> Hi,
>>>
>>> On 10/08/2015 11:18 AM, Jan Pazdziora wrote:
>>>> On Thu, Oct 08, 2015 at 11:12:37AM +0200, Oleg Fayans wrote:
>>>>>> When the ticket is addressed and these workarounds are no longer
>>>>>> needed -- what is our process for finding these workarounds and
>>>>>> reverting them, so that the tests test the real, expected behaviour?
>>>>> As per discussion with Martin Basti, it was decided that this 
>>>>> workaround
>>>>> will only be applied to the current 4-2 branch, not to the 
>>>>> upstream. In
>>>> That sounds like a reasonable plan for this issue.
>>>>
>>>>> upstream the issue itself will (supposedly) be solved
>>>> Except currently it's not, so (IIUIC) you will keep having
>>>> nondeterministic failures in master.
>>>>
>>>> I was mostly interested in the general approach that we have to
>>>> workarounds -- how do we track them, how do we make sure they don't
>>>> stick in tests forever, even after the issue was already properly
>>>> addressed.
>>>>
>>> That's actually a great point. I personally would like tickets to 
>>> have one more
>>> field: "workaround" containing the address of a workaround in the 
>>> format
>>> "path_to_the_file:line_number" or better even - a commit id of this 
>>> workaround,
>>> so that once the ticket is resolved, we could easily find what to 
>>> reverse.
>>>
>> Please don't add any more trac fields, there is too many of them 
>> already :-)
>> Keyword may serve better for now...
>>
> +1
>
> new trac field for a few workarounds per year is not worth it.
>
Perhaps we could use pytest's expected fail (xfail) or skip marker. [1] 
It would prevent test from failing in the report and once the underlying 
issue is fixed, it will raise as an unexpected pass.

It could be used as a temporary solution, once the issue is fixed, we 
would remove the mark from the test. This would probably need some 
workflow to be defined for these cases.

[1]: https://pytest.org/latest/skipping.html

-- 
Milan Kubik