[Freeipa-devel] Travis CI broke after merging PR 177

[Martin Babinsky]

Hi list,

https://github.com/freeipa/freeipa/pull/177 was recently merged despite 
causing nearly half of the tests in our Travis CI gating to fail. This 
broke Travis CI for all other PR that were rebased after this merge, 
causing false negative errors everywhere.

Fraser reverted the offending commits in 
https://github.com/freeipa/freeipa/pull/329 which restored Travis to 
original state (never mind PEP8 errors they were in the original code 
already).

Regarding this issues I have two questions:

a)

should I merge https://github.com/freeipa/freeipa/pull/329 and thus 
revert the breakage in order to unblock other contributors? Given the 
current traffic I think it is sufficient to wait for us to investigate 
and produce a fix. If not, please scream loudly.

b)

what can we improve to make the results of CI more visible to 
contributors? I think that I should sit down with Martin 2 and 
investigate the possibility to send notifications about negative CI 
results (sufficient IMO) to the mailing list.

In the meanwhile I would like to ask all reviewers to carefully check 
the output of failed Travis CI runs. If the job fails, you will see the 
results at the very end of the log. There are two sections: PEP8 errors 
and test output. You can expand both of them to see what went wrong and 
report it to the PR author if necessary.

The reviewer and author can then use the very same tool used in CI [1] 
to reproduce the failures locally. Using  '--no-cleanup' option during 
the run [2] leaves behind a running container which you can attach to 
and investigate further.

[1] https://github.com/freeipa/ipa-docker-test-runner
[2] https://github.com/freeipa/ipa-docker-test-runner/blob/master/README.md

If you have any additional questions/suggestions about Travis feel free 
to contact me.

-- 
Martin^3 Babinsky