RFC: Mass Review Process with Flags?

[Warren Togami]

You might have seen the "Merge Review" tickets beginning to hit 
fedora-package-review list.  The filing would be on-going now, but we 
hit a snag (found non-existent owners).  I am waiting for that to be 
corrected so this can proceed in an automated fashion.

Meanwhile, I hope we can figure out this aspect, because our decision of 
the review process influences how the rest of the reviews should be filed.

Bugzilla Ticket for Each Package
================================
We desire a paper trail to show that each package has gone through a 
review.  We want to be able to see who reviewed and approved each 
package.  We want to be able to refer to these tickets for historical 
purposes.

No Tracking Bugs
================
After mulling this over a bit, I have decided that using tracking bugs 
with this review would be too confusing, as well as too slow to be 
usable in a work-flow, and would generate too much extraneous mail spam. 
  This is simply TOO MANY bugs to work with in a single tracker.  Even 
if we made it more confusing by splitting into six trackers, it would 
still be slow and annoying.

Thankfully, it seems that Bugzilla flags should be a vastly superior 
alternative for tracking our mass review progress.

Review Flags!
=============
You may notice that all Fedora tickets now have the "fedora-review" flag 
if you are part of the fedora_bugs group in the Fedora Account System (FAS).

fedora-review BLANK (not reviewed yet)
fedora-review ? (review in progress)
fedora-review - (rejected with reason stated in comments)
fedora-review + (review approved)

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=225231
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=225232
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=225233

Please play with flipping the flags on throw-away test bugs like these. 
  If your Bugzilla account can't see flags, then you are not a member of 
the fedora_bugs group.  Please consider using your Bugzilla e-mail 
address in your Fedora Account System account so it is easier to sync up 
the permission.

What should ASSIGNED mean?
==========================
In the past we used ASSIGNED as "who the reviewer was".  But flags now 
give us the potential for something a bit more logical.

When a flag is set to any state -, ? or +, your name appears next to the 
flag.  If the flag state changed multiple times, you can view the bug's 
Activity Log to see who made those changes.

Querying Flags
==============
Query -> Advanced -> Boolean State
Flag -> is equal to -> fedora-review?
Flag -> is equal to -> fedora-review-
Flag -> is equal to -> fedora-review+
(or query for Package Review with a NOT of the three above to see 
everything with blank flags)

Canned queries can show useful info like this... although in the 
long-term we will want to use the Bugzilla xmlrpc.cgi interface to 
create a nicely formatted and cached static view elsewhere.  This static 
view can display far more bugs than the web interface.

Bug Mail Subjects
=================
Subject: APPROVED [Bug 225307] New: Merge Review: awesfx
Subject: TAKEN [Bug 225307] New: Merge Review: awesfx
Subject: REJECTED [Bug 225307] New: Merge Review: awesfx

When flag states change, if you would have received mail about it, then 
the mail subject is customized slightly to make it more obvious at a glance.

(dkl is not entirely thrilled by this part of the proposal because it 
would require a one-off ugly hack to Bugzilla.  However, if we decide 
that we think this part would be VERY useful, then we can request it 
anyway.)

(I suspect an UNTAKEN message is unnecessary, as well as grammatically 
incorrect?)

Proposal: Review Process Using Flags
====================================
1) Use flags exclusively for tracking the review state.  Flags state who 
set that flag, so there is no confusion.
2) The package owner is ASSIGNED instead of the reviewer.  This is more 
logically consistent than the past.

I believe this process works, but perhaps I missed something.  Thoughts?

Warren Togami
wtogami at redhat.com