Automate parts of review process?

[Tom 'spot' Callaway]

On Sun, 2006-12-31 at 05:33 +0100, Axel Thimm wrote:
> Hi,
> 
> I think large parts of the review process could be automated or at the
> very least aided, so that reviewers have less trivial stuff to check
> and can be reviewing more effectively.
> 
> In order to allow any kind of automation, the submission process would
> have to change, e.g. instead of posting a URL a packager would submit
> the whole src.rpm to some submission mechanism. There the system could
> test whether the package builds on all non-excluded platforms and on
> the supported distribution releases (development and any more the
> submitter choses to add). The system would furthermore be able to run
> rpmlint and other rpm checking tools on the generated packages and
> prepare a submission web page for the packager (for example if rpmlint
> output is not silent the packager would have to comment on it, or if a
> static lib is found the same and so on). The result would be cast into
> bugzilla.
> 
> That would catch most of the trivial packaging issues especially ones
> made by newbies before any reviewer needs to spend time on it.
> 
> Perhaps the review process itself could use a helper that for example
> offers the proper remaining checklists for the package in
> question. Since the helper has access to the package it would know
> which items of the review are irrelevant and hide them from the
> reviewer (e.g. if the package has no python bits mask away python
> related checks). This part would be optional, e.g. reviewers are free
> to use the usuall methods of checking a package or could opt to using
> the helper.
> 
> Would something like that make sense?

This has been proposed in the past, I even believe a few people
attempted to make some tools to achieve this "pre-review check". To some
extent, rpmlint falls into this (every review should have rpmlint run
against it).

I will repeat what I have said in the past: If someone writes a good,
easy to use, open source tool to check for Fedora Packaging Compliance
(tm), then I think we would consider having all new packages
automatically run through that tool before human-review. 

The submission process might not even have to change, it could be as
simple as having a daemon/script/cron tracking new bugzilla review
items, scraping the SRPM path out of it, building it in a plague
instance, running the tool, and returning the tool output (or plague
build failure) as a comment to the new ticket.

Short answer: Show me the tool. :)

~spot