Alpha/Beta/GA blocker criteria?

[Allen Kistler]

Adam Williamson wrote:
> There is a definition, but it's quite tough:
> 
> https://fedoraproject.org/wiki/QA/ReleaseCriteria
> 
> in practice, the net is cast somewhat wider at GA time, but we don't
> have a hard definition.
> 
> We've been trying to work to a hard definition at least for Alpha:
> considering bugs that significantly affect the critical path (see
> https://fedoraproject.org/wiki/Critical_Path_Packages_Proposal ) as
> blockers. The definition of 'significantly affect' is basically
> 'severity high or urgent according to
> https://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow#Severity ,
> but there are exceptions to this:
> https://bugzilla.redhat.com/show_bug.cgi?id=515410 is one, it's
> technically medium or even low severity, but practically it would cause
> severe surprise to _everyone_ installing the Alpha, so we consider it a
> blocker). To give a short definition without references: if it stops you
> being able to install, get into a graphical desktop, or update the
> system, it's a blocker bug.
> 
> We're trying to get to a decent solid public definition of blocker
> criteria for each release stage, but in practice it's very hard to write
> a policy that's both definitive yet sufficiently flexible to cover odd
> cases. I'm not sure we'll ever get all the way there, but we'll do our
> best.

So there's kind of two topics here.

First is release criteria, which ideally would be tied to development 
and the freezes.  Something like:
Alpha: no core high/urgent, no critical path or Gnome urgent.
Beta: no core medium/high/urgent, no critical path high/urgent, no Gnome 
urgent.
GA: no core medium/high/urgent, no cp medium/high/urgent, no Gnome 
high/urgent.
Bugs could always get fixed earlier, of course.

That's not anywhere close to a formal proposal.  I'm not trying to start 
a fire.  I'm just wondering aloud how to make the release criteria 
complementary to the freezes.  Plus I'd let the paid test lead try to 
sell it to the steering committee, anyway.

Second is 513104.  On the day of the alpha go/no-go meeting, I doubt it 
would be popular to add a bug to the alpha blocker.  So I'll just add it 
to the beta blocker.

Not just reusing ext2/ext3 filesystems (like /boot), but test cases like 
upgrading ext2/ext3 to ext4 should also fail, since they're not detected 
in the first place.  Or is it just me?