[Freeipa-devel] [PATCH] 0029 Check expected error messages in tests

[Petr Viktorin]

On 03/29/2012 11:17 PM, Rob Crittenden wrote:
> Petr Viktorin wrote:
>> On 03/26/2012 09:49 PM, Rob Crittenden wrote:
>>> Petr Viktorin wrote:
>>>> On 03/20/2012 01:39 PM, Petr Viktorin wrote:
>>>>> This patch adds checking error messages, not just types, to the
>>>>> XML-RPC
>>>>> tests.
>>>>> The checking is still somewhat hackish, since XML-RPC doesn't give us
>>>>> structured error info, but it should protect against regressions on
>>>>> issues like whether we put name or cli_name in a ValidationError.
>>>>>
>>>>> https://fedorahosted.org/freeipa/ticket/2549
>>>>>
>>>>
>>>> Updated and rebased to current master.
>>>
>>> NACK
>>>
>>> automember wrongly was testing for non-existent users rather than
>>> automember rules but those should still be tested IMHO, perhaps with
>>> both types.
>>>
>>> There is also some inconsistency. In host you use substitution to set
>>> the hostname in the error: '%s: host not found' % fqdn1 but in others
>>> (group, hostgroup for example) the name is hardcoded. I also noticed
>>> that some reasons are unicode and others are not.
>>>
>>> rob
>>
>> Added tests for automember, made all the reasons unicode, using
>> substitutions when variables are involved.
>>
>> The patch still only updates tests that didn't pass the error message
>> check.
>>
>
> I'm seeing three failures that I think are due to recently pushed
> patches. Otherwise it looks good.
>
> rob
>
>
> ======================================================================
> FAIL: test_netgroup[1]: netgroup_mod: Try to update non-existent
> u'netgroup1'
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in runTest
> self.test(*self.arg)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 247, in <lambda>
> func = lambda: self.check(nice, **test)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 260, in check
> self.check_exception(nice, cmd, args, options, expected)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 284, in check_exception
> assert_deepequal(expected.strerror, e.strerror)
> File "/home/rcrit/redhat/freeipa-beta1/tests/util.py", line 328, in
> assert_deepequal
> VALUE % (doc, expected, got, stack)
> AssertionError: assert_deepequal: expected != got.
>
> expected = u'netgroup1: netgroup not found'
> got = u'no such entry'
> path = ()
>
> ======================================================================
> FAIL: test_netgroup[4]: netgroup_add: Test an invalid nisdomain1 name
> u'domain1,domain2'
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in runTest
> self.test(*self.arg)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 247, in <lambda>
> func = lambda: self.check(nice, **test)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 260, in check
> self.check_exception(nice, cmd, args, options, expected)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 284, in check_exception
> assert_deepequal(expected.strerror, e.strerror)
> File "/home/rcrit/redhat/freeipa-beta1/tests/util.py", line 328, in
> assert_deepequal
> VALUE % (doc, expected, got, stack)
> AssertionError: assert_deepequal: expected != got.
>
> expected = u"invalid 'nisdomainname': may only include letters, numbers,
> _, - and ."
> got = u"invalid 'nisdomain': may only include letters, numbers, _, -,
> and ."
> path = ()
>
> ======================================================================
> FAIL: test_netgroup[5]: netgroup_add: Test an invalid nisdomain2 name
> u'+invalidnisdomain'
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/nose/case.py", line 187, in runTest
> self.test(*self.arg)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 247, in <lambda>
> func = lambda: self.check(nice, **test)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 260, in check
> self.check_exception(nice, cmd, args, options, expected)
> File
> "/home/rcrit/redhat/freeipa-beta1/tests/test_xmlrpc/xmlrpc_test.py",
> line 284, in check_exception
> assert_deepequal(expected.strerror, e.strerror)
> File "/home/rcrit/redhat/freeipa-beta1/tests/util.py", line 328, in
> assert_deepequal
> VALUE % (doc, expected, got, stack)
> AssertionError: assert_deepequal: expected != got.
>
> expected = u"invalid 'nisdomainname': may only include letters, numbers,
> _, - and ."
> got = u"invalid 'nisdomain': may only include letters, numbers, _, -,
> and ."
> path = ()
>

Updated patch attached.


-- 
Petr³
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-pviktori-0029-03-Fix-expected-error-messages-in-tests.patch
Type: text/x-patch
Size: 72239 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20120410/cf85ad91/attachment.bin>