[Freeipa-devel] Thoughts on tests (unit, integration, self-test)
Rich Megginson
rmeggins at redhat.com
Thu Feb 19 17:02:36 UTC 2009
Rob Crittenden wrote:
> Jason Gerard DeRose wrote:
>> This is a brain dump on all things related to the freeIPA tests included
>> in its source tree...
>>
>> One of my big goals with the Python code I've written for v2 is to make
>> freeIPA easier to test (especially easier to quickly test as you code,
>> while running everything in-tree). This is a challenging problem
>> because running full blown freeIPA requires some fairly invasive
>> configuration changes... you don't want to make these changes to your
>> workstation unless you're actually part of an IPA realm, and if you're
>> part of an IPA realm, you don't want to run these tests against (and
>> possibly break) a production realm.
>
> [ snip ]
>
> I generally agree with this approach, particularly when it comes to
> separating the integration tests from the unit tests. I like the idea
> of selftests but I haven't yet wrapped my mind around it. I keep
> thinking of Jurassic park where you are only testing what you are
> expecting and therefore all tests pass (even though there are raptors
> about to bite you).
>
> I think we should have 2 types of integration testing too: lite and
> full. Lite testing would utilize the python-litexml.py script and a
> full test would test against Apache running the XML-RPC module. In
> theory they should work the same way, this will confirm the theory. It
> should be as simple as changing a config option on the client so
> running one vs the other should be straightforward. It might be nice
> to be able to run this with a remote IPA server as well as a local one
> too.
>
> So I think you are on the right track here, we just need to flesh out
> what the SelfTest might look like. Would it essentially be one test
> per-method or would we be able to fake negative testing too (to test
> throwing exceptions)?
>
> I gather you envision this as separate make-test scripts (or make
> targets)?
>
> I'm a little less jaded when it comes to setting up an environment. I
> drop and re-create my IPA server on an almost daily basis it seems
> testing one thing or another. Running in a VM is a good idea though
> simply because IPA is so invasive in overwriting config files.
Another sort of half-way option - using mock to run tests. mock is
great if you need a full-blown OS file system with your packages
installed and you also need root access. mock can run any rhel or
fedora operating system (even 32-bit and 64-bit on a 64-bit machine).
mock is much more lightweight than running a VM. The only problem is
network connections e.g. you cannot have two directory servers running
at the same time both listening to port 389 - there are probably other
gotchas as well with hostnames (but some sort of /etc/hosts hack in the
chroot might be possible).
I have run the directory server acceptance test suite inside mock,
including such tests as 4 way MMR. It's pretty nifty to be able to use
mock to build and test packages in F-8, F-9, F-10, and even rhel
platforms, from a single machine with no VM.
>
> Great start.
>
> rob
>
> _______________________________________________
> Freeipa-devel mailing list
> Freeipa-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/freeipa-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3258 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20090219/4c102e9c/attachment.bin>
More information about the Freeipa-devel
mailing list