[Freeipa-devel] [PATCHES] 0230-0234 Integtration testing framework

Fri May 31 11:46:49 UTC 2013

Apply on top of my patches 0227-0234.

These patches add an initial integration testing framework.

Patch 0230 adds a plugin for ordered test classes.
Nose orders methods within a test suite alphabetically, but we generally 
want to run them in the order defined. This adds the @ordered decorator 
that causes Nose to do just that, provided the plugin is loaded and 
enabled, and that the methods are defined in the same file. The 
ipa-run-tests wrapper is changed to enable the plugin.
In the future we may want to use this for unit tests as well. It might 
also make sense to separate it from the FreeIPA project altogether.

Patch 0231 adds configuration for tests. This reads environment 
variables like:
- MASTER (FQDN of initial server)
- REPLICA (space-separated FQDNs of replicas)
- CLIENT (space-separated FQDNs of clients),
- IPATEST_DIR (directory the tests use on the remote machines)
etc., and loads them into an easy-to use Python object.
A tool called ipa-test-config is provided that generates a full set of 
environment variables for shell-based tests from these, either global or 
specific for a given host.
If environment variables don't work for us, alternate configuration 
methods can be added in the future.

Patch 0232 adds an integration test framework.
This extends Host object available from the configuration with methods 
to run commands and copy files on the remote host, and adds a base class 
for integration tests which can currently install and uninstall IPA in a 
"star" topology. (In the future, the install/uninstall code should also 
be made available as a shell command.)
A simple test for user replication between two masters is provided.
Log files from the remote hosts can be marked for collection, but the 
actual collection is left to a Nose plugin.
The base class uses the @ordered decorator mentioned above.

Patch 0233 improves on how commands are run on remote hosts.
In the previous patch, the process's stdin and stdout were combined as a 
quick hack to avoid the problem that if we first read stdout and then 
stderr, then stderr's buffer can fill up and we'd deadlock (and the 
other way around). With this patch the streams are read in parallel.
In the future this can be extended to calling whole commands in parallel 
(e.g. uninstalling IPA on all the hosts at once).

Patch 0234 adds log collection to the BeakerLib integration plugin.
This tars up the marked logs, downloads then, and calls rlFileSubmit on 
them.

---

Example procedure to run the integration tests:

Prepare two machines. Install IPA on them, enable root to SSH in, 
configure/disable firewall, apply current workarounds (RPM downgrading, 
SELinux), etc. (The plan is to provide no setup automation as part of 
the FreeIPA project, since systems/setups differ too much.)
I'll use "vm-116.idm.lab.bos.redhat.com" and 
"vm-089.idm.lab.bos.redhat.com" here.
The rest of the instructions apply to the "local" machine.

Install the RPMs.
On F19, remove special characters from /etc/fedora-release (BZ#700525).

Enter:
export MASTER=vm-116.idm.lab.bos.redhat.com
export REPLICA=vm-089.idm.lab.bos.redhat.com
export IPA_ROOT_SSH_PASSWORD=<your root pwd>
... and any extra config you might want ($FORWARDER, $DOMAIN, ..)

`ipa-test-config --global` should now print out something like:

export IPATEST_DIR='/root/ipatests'
export IPv6SETUP=''
export IPADEBUG=''
export IPA_ROOT_SSH_PASSWORD=<your root pwd>
export ADMINID='admin'
export ADMINPW='Secret123'
export ROOTDN='cn=Directory Manager'
export ROOTDNPWD='Secret123'
export DNSFORWARD='8.8.8.8'
export NISDOMAIN='ipatest'
export NTPSERVER='2.pool.ntp.org'
export DOMAIN_env1='idm.lab.bos.redhat.com'
export RELM_env1='IDM.LAB.BOS.REDHAT.COM'
export BASEDN_env1='dc=idm,dc=lab,dc=bos,dc=redhat,dc=com'
export MASTER_env1='vm-116.idm.lab.bos.redhat.com'
export BEAKERMASTER_env1='vm-116.idm.lab.bos.redhat.com'
export BEAKERMASTER_IP_env1='10.16.78.116'
export MASTER1_env1='vm-116.idm.lab.bos.redhat.com'
export BEAKERMASTER1_env1='vm-116.idm.lab.bos.redhat.com'
export BEAKERMASTER1_IP_env1='10.16.78.116'
export REPLICA_env1='vm-089.idm.lab.bos.redhat.com'
export BEAKERREPLICA_env1='vm-089.idm.lab.bos.redhat.com'
export BEAKERREPLICA_IP_env1='10.16.78.89'
export REPLICA1_env1='vm-089.idm.lab.bos.redhat.com'
export BEAKERREPLICA1_env1='vm-089.idm.lab.bos.redhat.com'
export BEAKERREPLICA1_IP_env1='10.16.78.89'
export CLIENT_env1=''
export BEAKERCLIENT_env1=''
export BEAKERCLIENT_IP_env1=''
export OTHER_env1=''
export BEAKEROTHER_env1=''
export BEAKEROTHER_IP_env1=''
export MASTER='vm-116.idm.lab.bos.redhat.com'
export BEAKERMASTER='vm-116.idm.lab.bos.redhat.com'
export MASTERIP='10.16.78.116'
export SLAVE='vm-089.idm.lab.bos.redhat.com'
export REPLICA='vm-089.idm.lab.bos.redhat.com'
export BEAKERSLAVE='vm-089.idm.lab.bos.redhat.com'
export SLAVEIP='10.16.78.89'

Now run:
( . /usr/share/beakerlib/beakerlib.sh
   rlJournalStart
   ipa-run-tests test_integration/test_simple_replication.py 
--with-beakerlib
   python -c 'print "="*100'
   rlJournalPrintText --full-journal
   rlJournalEnd
   )

The BeakerLib logging interleaved with Nose is a bit messy but better 
than nothing. Note that the output is buffered so 
installation/uninstallation may appear hung (use top on the remote host 
to see real-time activity).

Or you can run without BeakerLib:
ipa-run-tests test_integration/test_simple_replication.py
(Here the most time-consuming operations, setup & teardown, are not 
shown at all)

-- 
Petr³
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-pviktori-0230-Add-a-plugin-for-test-ordering.patch
Type: text/x-patch
Size: 4190 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20130531/17305a45/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-pviktori-0231-Add-a-framework-for-integration-test-configuration.patch
Type: text/x-patch
Size: 21290 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20130531/17305a45/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-pviktori-0232-Add-a-framework-for-integration-testing.patch
Type: text/x-patch
Size: 22703 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20130531/17305a45/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-pviktori-0233-Introduce-a-class-for-remote-commands.patch
Type: text/x-patch
Size: 10810 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20130531/17305a45/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-pviktori-0234-Collect-logs-from-tests.patch
Type: text/x-patch
Size: 7020 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20130531/17305a45/attachment-0004.bin>