[Freeipa-devel] Continuous Integration & dependency tree testing

[Martin Kosek]

On 09/23/2014 08:39 PM, Petr Spacek wrote:
> Hello,
> 
> The recent problem with Tomcat forced me to think how we can detect that some other package broke IPA before it hit users.
> 
> After all, it seems pretty easy.
> 
> 0) Get a VM for testing purposes (preferably with minimal set of IPA dependencies). The VM has to be reverted to snapshot after every run. (This is pretty simple with RHEV-M/libvirt.)
> 
> 1) Detect all changes in the minimal set of packages installed on the test VM.
> 
> It is easy because we can simply hash output of YUM/DNF "upgrade" command:
> 
> # echo "n" | dnf upgrade | md5sum > changeset.id
> 
> changeset.id will change every time when a package which is present on test system changes but stays the same if there were no updates from last run. (There could be false positives but it doesn't matter - the test will be simply run twice.)
> 
> 2) if (changeset.id != previous_changeset.id)
> run_tests()
> 
> 3) This machinery could be triggered by cron or even by a message on fedmsg about a new package pushed to updates-testing...
> 
> Tests could use released versions of IPA (present in fedora-updates) to get better signal/noise ratio.
> 
> Okay, it is time to stop dreaming and go to bed. Good night from Europe :-)

The current answer to this is current FreeIPA Jenkins CI instance. It is run on
top of updates-testing, so as soon as some breaking package gets in
updates-testing, the job should break.

There is a good question though how the test should be triggered when our
dependency changes and the code does not (especially in stable branches). This
may be the part where the mechanism you propose would come in play.

Martin