RFC: fedora.us QA approval format

[Aurelien Bompard]

Toshio wrote:

> New thoughts:
> * I like bullets for lists of things.  Helps to delimit separate entries
> on a page.

Agreed. I like that too :)

> * I'd leave the src.rpm under MD5Sums so if the hash algorithm gets
> updated a program will parse the type of hash before encountering its
> first hash.

Agreed, that's a good point. 

> * I'd put the recommendation (PUBLISH +1| NEEDSWORK) as the first line.
> It's the most important thing to pick out of the clutter of the review.
> All the rest is "supporting evidence" of the recommendation.

OK, good point too.

> * What are your thoughts on which parts of the review will be used by
> automated tools?  I would say the Recommendation and MD5Sum section are
> definites.  The Sources section a maybe, and the rest probably not.

This looks reasonable to me. The mandatory lines are already listed on
PackageSubmissionQAPolicy:
--8<------
In cases where a QA message would lead to package approval or publication,
the message MUST be GPG clearsigned, and contain md5sums of the reviewed
source RPM as well as preferably all upstream sources.
--8<------

So that's as you wrote: recommandation and MD5SUMs. The Sources section
seems not to be mandatory, but it can do no harm if it is easily parseable.
On that point I agree on a "OK" + any explanation. NOT OK and N/A look fine
to me too.

> If an entry isn't going to be machine parsed,
> then I'd stick it into a Good: vs NEEDSWORK: section and forgo the
> (OK|NOT OK)

OK by me. In this case, since it is not going to be parsed, the reviewer is
totally free to choose his prose ;-) The Good and Needswork sections are
clearer and should be respected though.
 
> * It doesn't show up in this review -- I'd put any patches in the
> Sources section, would you agree?

Agreed, they are more or less like a local source anyway.
 
> Here's my take on what I'd like a review to look like
> http://www.fedora.us/wiki/QAFormatAlt

Looks good to me. I'd remove the blank line between the srpm's md5sum and
the rest though. Since it's the first one anyway, we can save space there.
But that's totally cosmetic, and if you like it a lot, we'll keep it. (it's
just to have something to improve on your version ;-p )
I'm going to merge that into QAFormat.

> This assumes that automated tools aren't going to check whether Package
> Name/SRPM Signature/etc were verified.  If you think they should be then
> I'd take your example with a header (ex: Essential Checks) and bullets
> and normalize all the entries to OK instead of OK, VALID, and YES.

The question actually is: Should the automated tool control that the
approval have the minimum required items, or should it also check that the
QA Checklist has been gone through. Because of the lenght and the nature of
the QA Checklist, I think it is impossible to make sure that the QA check
was actually correctly done. I think we have to trust a bit the reviewer,
and that's why we need at least two QA approvals.
So I don't mind having the tool check only the minimum, and having an human
eye deciding if the QA check was correctly done. OTOH, I'm not the one who
will be operating the tool, and I'm not the one doing the job. If the
release managers want the (yet-to-be-written) tool to be more automatic,
and thus to remove more of their load, they sould impose a parseable format
for the rest of the checklist too. Can any of the release managers tell us
a bit about that please ?

Thanks a lot for your comments Toshio, I'm happy that we're reaching a
satisfying QA template.

Bye,

Aurélien
-- 
http://gauret.free.fr   ~~~~   Jabber : gauret at amessage.info
"First they ignore you, then they laugh at you, then they fight you,
then you win. " -- Mahatma Ghandi               ^^^^^^^^^^^^^^^^^^^
                                                   Linux is here.