[Freeipa-devel] Automated Fedora update testing

[Petr Vobornik]

On 04/29/2017 02:07 AM, Adam Williamson wrote:
> Hi folks! I thought this might be of interest to the FreeIPA community,
> so I thought I'd write it up here in case anyone missed it elsewhere.
>
<snip>
>
> Until recently we ran these tests only on Fedora's nightly development
> release distribution composes. Recently, though, we deployed some
> enhancements to our openQA setup that let us run tests on Fedora
> distribution updates as well, and have the results made visible through
> the Fedora update system (Bodhi). The tests are automatically run on
> any critical path package, and as of today, they are also run on any
> update containing any of a manually-tended list of FreeIPA-related
> packages:
>
> 389-ds
> 389-ds-base
> bind
> bind-dyndb-ldap
> certmonger
> ding-libs
> freeipa
> krb5-server
> pki-core
> sssd
> tomcat
> cockpit
>
> This means that for any Fedora update containing one of these or any
> critical path package, Fedora's openQA FreeIPA tests should run, and
> you should see the results in the Fedora update system (Bodhi). You can
> see the results in Bodhi by clicking the Automated Updates tab for any
> update. For instance, here's a recent 389-ds-base update for Fedora 26:
>
> https://bodhi.fedoraproject.org/updates/FEDORA-2017-15e2a038b2
>
> If you look at the Automated Tests tab, you can see passes for:
>
> update.server_role_deploy_domain_controller
> update.realmd_join_cockpit
> update.realmd_join_sssd
>
> indicating that this update didn't cause any problems for FreeIPA.
> Clicking on any test result will take you to the openQA page for the
> test, where you can diagnose failures and so on (explaining how to do
> this is a bit beyond the scope of this mail, please do ask me if you're
> interested!)

This is really great.

>
> I hope this stuff will help us avoid shipping updates that break
> FreeIPA (and other key components). If you have any questions,
> concerns, comments, or suggestions, please do ask!
> .
>
> Note, if you're interested in the results for the nightly Fedora
> distribution composes, an email summary of the results for those is
> sent each time they're run to the Fedora test@ and devel@ lists, look
> for mails with "compose check report" in the subject. Any time any of
> the FreeIPA tests fails, the failure will be listed in the mail (passed
> tests are not specifically listed, just a count of them). I usually
> keep an eye on those results and analyze failures and file bugs,
> though.
>

Is there a way now to check current state of current Fedoras 
automatically using a script - e.g. avoid parsing mailing list or going 
through runs in OpenQA?

-- 
Petr Vobornik