[Freeipa-devel] CI: exporting test runner output
Fraser Tweedale
ftweedal at redhat.com
Thu Jan 5 08:54:29 UTC 2017
On Thu, Jan 05, 2017 at 09:38:03AM +0100, Tomas Krizek wrote:
> On 01/05/2017 09:25 AM, Fraser Tweedale wrote:
> > On Thu, Jan 05, 2017 at 08:53:14AM +0100, Martin Babinsky wrote:
> >> On 01/05/2017 08:06 AM, Fraser Tweedale wrote:
> >>> Hi all,
> >>>
> >>> Although it has been discussed before and met with some skepticism,
> >>> here is a POC that exporting test runner output to, e.g. a pastebin,
> >>> does work:
> >>>
> >>> - experimental commit: https://github.com/freeipa/freeipa/pull/370
> >>> - example paste: https://paste.fedoraproject.org/520085/
> >>> (it is gzipped for reasons discussed in the PR)
> >>>
> >>> I think we should proceed with getting these artifacts out of Travis
> >>> and stored somewhere (it doesn't have to be
> >>> paste.fedoraproject.org). ``tail -n 5000`` of the log file has
> >>> proven to be not enough to diagnose all failures.
> >>>
> >> Wow this is great, why have I not thought about it beforehand?
> Seems like a great feature. Thanks, Fraser!
> >> We can reduce the log size if we truncate everything before ERRORS/FAILURES
> >> output of pytest run (we leave the log as it is if the fail occurs before
> >> this stage), that should shave off considerable amount of cruft from the
> >> paste unless somebody sends a PR that breaks all out tests :D.
> >>
> >>> If we stick with paste.fedoraproject.org, we can send to a
> >>> "project-specific" namespace e.g.
> >>> https://paste.fedoraproject.org/~freeipa, so that we do not clutter
> >>> up the main archive (I think).
> >>>
I was wrong. All "project" pastes appear in main namespace as well
as project namespace. Not sure if by design or not.
> >>> A few questions for discussion:
> >>>
> >>> 1. Stick with fpaste or not? If so, use "~freeipa" namespace?
> >>> (Keep in mind that the size limitation that exists for fpaste,
> >>> which requires compressing the artifact, may not be a problem
> >>> elsewhere).
> >>>
> >>> 2. Export log always, or only if the build job failed?
> >>>
> >> I would also paste the output to "freeipa" or even better "freeipa-travis"
> >> namespace and only send it if the job fails.
> >>
> > I might go with "freeipa-ci".
> +1
>
Unfortunately fpaste can't handle this. Has to be all-alpha. So we
can use "freeipaci" but given the constraint I would rather just use
"freeipa". I shall file a fedora-infra ticket to see if this can be
addressed.
> >>> 3. Should pasted logs expire? If so, what should TTL be?
> >>>
> >> IMHO yes, but TTL is hard to determine, since the author of the PR may not
> >> be present to review the results immediately (because he is on PTO etc.). I
> >> think we should set TTL to something like 1 week and as a fallback keep
> >> tailing the CI results log.
> >>
> > 1 week sounds reasonable. We can change it later if we need to.
> I actually wouldn't mind extending this to something like 2-4 weeks. In
> some cases it might be useful to have access to older logs (PTOs, or
> simply to just view the history for some reason). Is there any downside
> to keeping the logs for a bit longer?
>
Not really. I was thinking server diskspace is logs were very big
but now that we're compressing I don't think it matters. 4 weeks,
sure why not :)
> >>> 4. Should we continue to `tail -n 5000` the log as we currently do,
> >>> or just rely on exported log?
> If you're talking about the log in the travis web interface, I would
> keep it. It's easily accessible from the browser.
> >>> Thanks,
> >>> Fraser
> >> Fraser, are you OK with waiting with this effort until we push
> >> https://github.com/freeipa/freeipa/pull/361 ?. I will just do some more
> >> adjustments there (like result log trimming) and it should be pushed ASAP.
> >>
> > Yes, I was aware that there would be conflicts with this PR. I
> > don't mind waiting. Thanks for your input.
> >
> > Cheers,
> > Fraser
> >
> --
> Tomas Krizek
>
>
More information about the Freeipa-devel
mailing list