[Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

[Oleg Fayans]

And here is how the run looks like:

$ ipa-run-tests test_integration/test_topology.py
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] 
Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission 
denied: 'yacctab.py'
==================================================================================== 
test session starts 
=====================================================================================
platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
plugins: sourceorder-0.5, multihost-1.0
collected 3 items

test_integration/test_topology.py ..x

=========================================================================== 
2 passed, 1 xfailed in 1558.66 seconds 
===========================================================================



On 08/12/2016 04:05 PM, Martin Basti wrote:
>
>
> On 12.08.2016 15:48, Oleg Fayans wrote:
>> Hi Martin,
>>
>>
>>
>> On 08/11/2016 10:05 AM, Martin Basti wrote:
>>>
>>>
>>> On 10.08.2016 20:32, Oleg Fayans wrote:
>>>>
>>>>
>>>>
>>> Hello,
>>>
>>> before we jump into fixing tests, my question is: Was this planned
>>> change and not reflected by test, or switched values are unwanted side
>>> effect and thus bug for us?
>>
>> That's a marvelous question! The test used to pass, which means that
>> at some point the convention of naming the segments must have changed.
>> Is it a bug? I do not think so: the feature still works as expected.
>
> Ludwig, do you know details about this change, why positions of server
> names are different than used to be in topology name?
>
>>
>>>
>>> Ticket contains almost no info, except a traceback and it says nothing.
>>> Commit message says at least something.
>>>
>>> I'm not sure if this patch fixes that ticket, because traceback in test
>>> shows error message that "removal of segment will disconnect topology",
>>> but this patch only swap order of replica names in segment name. I would
>>> expect that you should get different error, something like segment does
>>> not exist.
>> Which I do get in jenkins job N 37: "segment not found"
>>
>> In fact, the error in the issue is unrelated to the fix, you are right.
>
>> To tell the truth, I just put a random error from one of the jenkins
>> topology testruns into the issue.
> This is very good way how to report tickets:
> * nobody knows what happened
> * nobody can search in current tickets,  what is wrong without proper
> description
> * developers cannot investigate issue, because there is even no name of
> exact test in ticket, no steps to reproduce, nothing
> * without proper tickets it is hard to backport patches correctly, if
> patch fixes different issue than is reported
>
> I'm closing ticket as invalid, please follow
> http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
> proper ticket.
>
>> This particular error message was caused by a previous replica
>> installation failure, which resulted in existing only one segment
>> instead of three:
>> master <-> replica1
>> instead of:
>> master <-> replica1,
>> master <-> replica2
>> replica1 <-> replica2
>>
>> In fact the patch supplied fixes 2 tests at once:
>> The first test tries to remove the unexisting segment master <->
>> replica2 and fails, the second test expects the line topology
>> master <-> replica1 <-> replica2.
>> It removes the connection between replica1 and replica2, expects the
>> operation to fail but it does not because the connection between
>> master and replica2 exists
>>
>> the output from the testrun with the patch applied:
>>
>>
>> -bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
>> WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
>> Permission denied: 'lextab.py'
>> WARNING: yacc table file version is out of date
>> WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
>> denied: 'yacctab.py'
>> ====================================================================================
>> test session starts
>> =====================================================================================
>>
>> platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
>> plugins: sourceorder-0.5, multihost-1.0
>> collected 3 items
>>
>> test_integration/test_topology.py ...
>>
>> ================================================================================
>> 3 passed in 2156.82 seconds
>> =================================================================================
>>
>>
>
> I don't care about test output until there is no valid description of
> problem, fixing test may just cover real issue.
> Martin^2
>>>
>>> Martin^2
>>>
>>>
>>
>

-- 
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.