[Freeipa-devel] Travis CI broke after merging PR 177

[Standa Laznicka]

On 12/13/2016 09:41 AM, Martin Babinsky wrote:
> Hi list,
>
> https://github.com/freeipa/freeipa/pull/177 was recently merged 
> despite causing nearly half of the tests in our Travis CI gating to 
> fail. This broke Travis CI for all other PR that were rebased after 
> this merge, causing false negative errors everywhere.
>
> Fraser reverted the offending commits in 
> https://github.com/freeipa/freeipa/pull/329 which restored Travis to 
> original state (never mind PEP8 errors they were in the original code 
> already).
>
> Regarding this issues I have two questions:
>
> a)
>
> should I merge https://github.com/freeipa/freeipa/pull/329 and thus 
> revert the breakage in order to unblock other contributors? Given the 
> current traffic I think it is sufficient to wait for us to investigate 
> and produce a fix. If not, please scream loudly.
>
> b)
>
> what can we improve to make the results of CI more visible to 
> contributors? I think that I should sit down with Martin 2 and 
> investigate the possibility to send notifications about negative CI 
> results (sufficient IMO) to the mailing list.
>
> In the meanwhile I would like to ask all reviewers to carefully check 
> the output of failed Travis CI runs. If the job fails, you will see 
> the results at the very end of the log. There are two sections: PEP8 
> errors and test output. You can expand both of them to see what went 
> wrong and report it to the PR author if necessary.
>
> The reviewer and author can then use the very same tool used in CI [1] 
> to reproduce the failures locally. Using  '--no-cleanup' option during 
> the run [2] leaves behind a running container which you can attach to 
> and investigate further.
>
> [1] https://github.com/freeipa/ipa-docker-test-runner
> [2] 
> https://github.com/freeipa/ipa-docker-test-runner/blob/master/README.md
>
> If you have any additional questions/suggestions about Travis feel 
> free to contact me.
>
Hello,

I would personally wait for a proper fix in the a) case. We've been able 
to perform well even without Travis up until recently plus 99% of the 
pull requests are created and reviewed by team members who are aware of 
the situation.

as for b): I can't quite imagine how sending mails to the devel list 
would work but I assume you'd come up with some kind of sane solution.

Standa