[Freeipa-devel] Thoughts on tests (unit, integration, self-test)

[Rich Megginson]

Rob Crittenden wrote:
> Rich Megginson wrote:
>> Rob Crittenden wrote:
>>> Jason Gerard DeRose wrote:
>>>> This is a brain dump on all things related to the freeIPA tests 
>>>> included
>>>> in its source tree...
>>>>
>>>> One of my big goals with the Python code I've written for v2 is to 
>>>> make
>>>> freeIPA easier to test (especially easier to quickly test as you code,
>>>> while running everything in-tree).  This is a challenging problem
>>>> because running full blown freeIPA requires some fairly invasive
>>>> configuration changes... you don't want to make these changes to your
>>>> workstation unless you're actually part of an IPA realm, and if you're
>>>> part of an IPA realm, you don't want to run these tests against (and
>>>> possibly break) a production realm.
>>>
>>> [ snip ]
>>>
>>> I generally agree with this approach, particularly when it comes to 
>>> separating the integration tests from the unit tests. I like the 
>>> idea of selftests but I haven't yet wrapped my mind around it. I 
>>> keep thinking of Jurassic park where you are only testing what you 
>>> are expecting and therefore all tests pass (even though there are 
>>> raptors about to bite you).
>>>
>>> I think we should have 2 types of integration testing too: lite and 
>>> full. Lite testing would utilize the python-litexml.py script and a 
>>> full test would test against Apache running the XML-RPC module. In 
>>> theory they should work the same way, this will confirm the theory. 
>>> It should be as simple as changing a config option on the client so 
>>> running one vs the other should be straightforward. It might be nice 
>>> to be able to run this with a remote IPA server as well as a local 
>>> one too.
>>>
>>> So I think you are on the right track here, we just need to flesh 
>>> out what the SelfTest might look like. Would it essentially be one 
>>> test per-method or would we be able to fake negative testing too (to 
>>> test throwing exceptions)?
>>>
>>> I gather you envision this as separate make-test scripts (or make 
>>> targets)?
>>>
>>> I'm a little less jaded when it comes to setting up an environment. 
>>> I drop and re-create my IPA server on an almost daily basis it seems 
>>> testing one thing or another. Running in a VM is a good idea though 
>>> simply because IPA is so invasive in overwriting config files.
>> Another sort of half-way option - using mock to run tests.  mock is 
>> great if you need a full-blown OS file system with your packages 
>> installed and you also need root access.  mock can run any rhel or 
>> fedora operating system (even 32-bit and 64-bit on a 64-bit 
>> machine).  mock is much more lightweight than running a VM.  The only 
>> problem is network connections e.g. you cannot have two directory 
>> servers running at the same time both listening to port 389 - there 
>> are probably other gotchas as well with hostnames (but some sort of 
>> /etc/hosts hack in the chroot might be possible).
>>
>> I have run the directory server acceptance test suite inside mock, 
>> including such tests as 4 way MMR.  It's pretty nifty to be able to 
>> use mock to build and test packages in F-8, F-9, F-10, and even rhel 
>> platforms, from a single machine with no VM.
>>
>
> I actually thought about that but figured it would be too much of a 
> pain to set up. Can you provide any details/tips?
https://fedoraproject.org/wiki/Projects/Mock - basic info and links to 
more detailed info
You'd basically do something like this:
edit the mock config (/etc/mock/*.cfg) to add a local yum repo e.g. 
file:///my/local/repo - copy the rpms you want to install and test into 
this repo and do a createrepo
mock -r platform --install packages you want to test
mock -r platform --shell -- sh -c /path/inside/chroot/testScript
>
> thanks
>
> rob

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3258 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20090219/da5bb828/attachment.bin>