[Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

[Oleg Fayans]

Again ping for review, please it completely blocks the whole job.

On 09/07/2016 03:27 PM, Oleg Fayans wrote:
> ping for review
>
> On 08/24/2016 01:58 PM, Oleg Fayans wrote:
>> And here is how the run looks like:
>>
>> $ ipa-run-tests test_integration/test_topology.py
>> WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
>> Permission denied: 'lextab.py'
>> WARNING: yacc table file version is out of date
>> WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
>> denied: 'yacctab.py'
>> ====================================================================================
>>
>> test session starts
>> =====================================================================================
>>
>>
>> platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
>> plugins: sourceorder-0.5, multihost-1.0
>> collected 3 items
>>
>> test_integration/test_topology.py ..x
>>
>> ===========================================================================
>>
>> 2 passed, 1 xfailed in 1558.66 seconds
>> ===========================================================================
>>
>>
>>
>>
>> On 08/12/2016 04:05 PM, Martin Basti wrote:
>>>
>>>
>>> On 12.08.2016 15:48, Oleg Fayans wrote:
>>>> Hi Martin,
>>>>
>>>>
>>>>
>>>> On 08/11/2016 10:05 AM, Martin Basti wrote:
>>>>>
>>>>>
>>>>> On 10.08.2016 20:32, Oleg Fayans wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>> Hello,
>>>>>
>>>>> before we jump into fixing tests, my question is: Was this planned
>>>>> change and not reflected by test, or switched values are unwanted side
>>>>> effect and thus bug for us?
>>>>
>>>> That's a marvelous question! The test used to pass, which means that
>>>> at some point the convention of naming the segments must have changed.
>>>> Is it a bug? I do not think so: the feature still works as expected.
>>>
>>> Ludwig, do you know details about this change, why positions of server
>>> names are different than used to be in topology name?
>>>
>>>>
>>>>>
>>>>> Ticket contains almost no info, except a traceback and it says
>>>>> nothing.
>>>>> Commit message says at least something.
>>>>>
>>>>> I'm not sure if this patch fixes that ticket, because traceback in
>>>>> test
>>>>> shows error message that "removal of segment will disconnect
>>>>> topology",
>>>>> but this patch only swap order of replica names in segment name. I
>>>>> would
>>>>> expect that you should get different error, something like segment
>>>>> does
>>>>> not exist.
>>>> Which I do get in jenkins job N 37: "segment not found"
>>>>
>>>> In fact, the error in the issue is unrelated to the fix, you are right.
>>>
>>>> To tell the truth, I just put a random error from one of the jenkins
>>>> topology testruns into the issue.
>>> This is very good way how to report tickets:
>>> * nobody knows what happened
>>> * nobody can search in current tickets,  what is wrong without proper
>>> description
>>> * developers cannot investigate issue, because there is even no name of
>>> exact test in ticket, no steps to reproduce, nothing
>>> * without proper tickets it is hard to backport patches correctly, if
>>> patch fixes different issue than is reported
>>>
>>> I'm closing ticket as invalid, please follow
>>> http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
>>> proper ticket.
>>>
>>>> This particular error message was caused by a previous replica
>>>> installation failure, which resulted in existing only one segment
>>>> instead of three:
>>>> master <-> replica1
>>>> instead of:
>>>> master <-> replica1,
>>>> master <-> replica2
>>>> replica1 <-> replica2
>>>>
>>>> In fact the patch supplied fixes 2 tests at once:
>>>> The first test tries to remove the unexisting segment master <->
>>>> replica2 and fails, the second test expects the line topology
>>>> master <-> replica1 <-> replica2.
>>>> It removes the connection between replica1 and replica2, expects the
>>>> operation to fail but it does not because the connection between
>>>> master and replica2 exists
>>>>
>>>> the output from the testrun with the patch applied:
>>>>
>>>>
>>>> -bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
>>>> WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
>>>> Permission denied: 'lextab.py'
>>>> WARNING: yacc table file version is out of date
>>>> WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
>>>> denied: 'yacctab.py'
>>>> ====================================================================================
>>>>
>>>>
>>>> test session starts
>>>> =====================================================================================
>>>>
>>>>
>>>>
>>>> platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
>>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
>>>> plugins: sourceorder-0.5, multihost-1.0
>>>> collected 3 items
>>>>
>>>> test_integration/test_topology.py ...
>>>>
>>>> ================================================================================
>>>>
>>>>
>>>> 3 passed in 2156.82 seconds
>>>> =================================================================================
>>>>
>>>>
>>>>
>>>>
>>>
>>> I don't care about test output until there is no valid description of
>>> problem, fixing test may just cover real issue.
>>> Martin^2
>>>>>
>>>>> Martin^2
>>>>>
>>>>>
>>>>
>>>
>>
>

-- 
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.