[Freeipa-devel] [Test][patch-0058] Fixed topology tests failures in CI

[Martin Basti]

On 14.09.2016 10:43, Oleg Fayans wrote:
> Again ping for review, please it completely blocks the whole job.
>
> On 09/07/2016 03:27 PM, Oleg Fayans wrote:
>> ping for review
>>
>> On 08/24/2016 01:58 PM, Oleg Fayans wrote:
>>> And here is how the run looks like:
>>>
>>> $ ipa-run-tests test_integration/test_topology.py
>>> WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
>>> Permission denied: 'lextab.py'
>>> WARNING: yacc table file version is out of date
>>> WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
>>> denied: 'yacctab.py'
>>> ==================================================================================== 
>>>
>>>
>>> test session starts
>>> ===================================================================================== 
>>>
>>>
>>>
>>> platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: pytest.ini
>>> plugins: sourceorder-0.5, multihost-1.0
>>> collected 3 items
>>>
>>> test_integration/test_topology.py ..x
>>>
>>> =========================================================================== 
>>>
>>>
>>> 2 passed, 1 xfailed in 1558.66 seconds
>>> =========================================================================== 
>>>
>>>
>>>
>>>
>>>
>>> On 08/12/2016 04:05 PM, Martin Basti wrote:
>>>>
>>>>
>>>> On 12.08.2016 15:48, Oleg Fayans wrote:
>>>>> Hi Martin,
>>>>>
>>>>>
>>>>>
>>>>> On 08/11/2016 10:05 AM, Martin Basti wrote:
>>>>>>
>>>>>>
>>>>>> On 10.08.2016 20:32, Oleg Fayans wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> before we jump into fixing tests, my question is: Was this planned
>>>>>> change and not reflected by test, or switched values are unwanted 
>>>>>> side
>>>>>> effect and thus bug for us?
>>>>>
>>>>> That's a marvelous question! The test used to pass, which means that
>>>>> at some point the convention of naming the segments must have 
>>>>> changed.
>>>>> Is it a bug? I do not think so: the feature still works as expected.
>>>>
>>>> Ludwig, do you know details about this change, why positions of server
>>>> names are different than used to be in topology name?
>>>>
>>>>>
>>>>>>
>>>>>> Ticket contains almost no info, except a traceback and it says
>>>>>> nothing.
>>>>>> Commit message says at least something.
>>>>>>
>>>>>> I'm not sure if this patch fixes that ticket, because traceback in
>>>>>> test
>>>>>> shows error message that "removal of segment will disconnect
>>>>>> topology",
>>>>>> but this patch only swap order of replica names in segment name. I
>>>>>> would
>>>>>> expect that you should get different error, something like segment
>>>>>> does
>>>>>> not exist.
>>>>> Which I do get in jenkins job N 37: "segment not found"
>>>>>
>>>>> In fact, the error in the issue is unrelated to the fix, you are 
>>>>> right.
>>>>
>>>>> To tell the truth, I just put a random error from one of the jenkins
>>>>> topology testruns into the issue.
>>>> This is very good way how to report tickets:
>>>> * nobody knows what happened
>>>> * nobody can search in current tickets,  what is wrong without proper
>>>> description
>>>> * developers cannot investigate issue, because there is even no 
>>>> name of
>>>> exact test in ticket, no steps to reproduce, nothing
>>>> * without proper tickets it is hard to backport patches correctly, if
>>>> patch fixes different issue than is reported
>>>>
>>>> I'm closing ticket as invalid, please follow
>>>> http://www.chiark.greenend.org.uk/~sgtatham/bugs.html and file a new
>>>> proper ticket.
>>>>
>>>>> This particular error message was caused by a previous replica
>>>>> installation failure, which resulted in existing only one segment
>>>>> instead of three:
>>>>> master <-> replica1
>>>>> instead of:
>>>>> master <-> replica1,
>>>>> master <-> replica2
>>>>> replica1 <-> replica2
>>>>>
>>>>> In fact the patch supplied fixes 2 tests at once:
>>>>> The first test tries to remove the unexisting segment master <->
>>>>> replica2 and fails, the second test expects the line topology
>>>>> master <-> replica1 <-> replica2.
>>>>> It removes the connection between replica1 and replica2, expects the
>>>>> operation to fail but it does not because the connection between
>>>>> master and replica2 exists
>>>>>
>>>>> the output from the testrun with the patch applied:
>>>>>
>>>>>
>>>>> -bash-4.3$ ipa-run-tests test_integration/test_topology.py --pdb
>>>>> WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13]
>>>>> Permission denied: 'lextab.py'
>>>>> WARNING: yacc table file version is out of date
>>>>> WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission
>>>>> denied: 'yacctab.py'
>>>>> ==================================================================================== 
>>>>>
>>>>>
>>>>>
>>>>> test session starts
>>>>> ===================================================================================== 
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> platform linux2 -- Python 2.7.11, pytest-2.9.2, py-1.4.31, 
>>>>> pluggy-0.3.1
>>>>> rootdir: /usr/lib/python2.7/site-packages/ipatests, inifile: 
>>>>> pytest.ini
>>>>> plugins: sourceorder-0.5, multihost-1.0
>>>>> collected 3 items
>>>>>
>>>>> test_integration/test_topology.py ...
>>>>>
>>>>> ================================================================================ 
>>>>>
>>>>>
>>>>>
>>>>> 3 passed in 2156.82 seconds
>>>>> ================================================================================= 
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> I don't care about test output until there is no valid description of
>>>> problem, fixing test may just cover real issue.
>>>> Martin^2
>>>>>>
>>>>>> Martin^2
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

ACK

master:
* 49fbbb0641df2adab28fd3440686cb7430645c85 Fixed segment naming in 
topology tests
* 3e4740f788aee00ae03a61d39238f605779fcece Xfailed a test that fails due 
to 6250