[Freeipa-devel] CI: exporting test runner output
Martin Babinsky
mbabinsk at redhat.com
Thu Jan 5 07:53:14 UTC 2017
On 01/05/2017 08:06 AM, Fraser Tweedale wrote:
> Hi all,
>
> Although it has been discussed before and met with some skepticism,
> here is a POC that exporting test runner output to, e.g. a pastebin,
> does work:
>
> - experimental commit: https://github.com/freeipa/freeipa/pull/370
> - example paste: https://paste.fedoraproject.org/520085/
> (it is gzipped for reasons discussed in the PR)
>
> I think we should proceed with getting these artifacts out of Travis
> and stored somewhere (it doesn't have to be
> paste.fedoraproject.org). ``tail -n 5000`` of the log file has
> proven to be not enough to diagnose all failures.
>
Wow this is great, why have I not thought about it beforehand?
We can reduce the log size if we truncate everything before
ERRORS/FAILURES output of pytest run (we leave the log as it is if the
fail occurs before this stage), that should shave off considerable
amount of cruft from the paste unless somebody sends a PR that breaks
all out tests :D.
> If we stick with paste.fedoraproject.org, we can send to a
> "project-specific" namespace e.g.
> https://paste.fedoraproject.org/~freeipa, so that we do not clutter
> up the main archive (I think).
>
> A few questions for discussion:
>
> 1. Stick with fpaste or not? If so, use "~freeipa" namespace?
> (Keep in mind that the size limitation that exists for fpaste,
> which requires compressing the artifact, may not be a problem
> elsewhere).
>
> 2. Export log always, or only if the build job failed?
>
I would also paste the output to "freeipa" or even better
"freeipa-travis" namespace and only send it if the job fails.
> 3. Should pasted logs expire? If so, what should TTL be?
>
IMHO yes, but TTL is hard to determine, since the author of the PR may
not be present to review the results immediately (because he is on PTO
etc.). I think we should set TTL to something like 1 week and as a
fallback keep tailing the CI results log.
> 4. Should we continue to `tail -n 5000` the log as we currently do,
> or just rely on exported log?
>
> Thanks,
> Fraser
>
Fraser, are you OK with waiting with this effort until we push
https://github.com/freeipa/freeipa/pull/361 ?. I will just do some more
adjustments there (like result log trimming) and it should be pushed ASAP.
--
Martin^3 Babinsky
More information about the Freeipa-devel
mailing list