"What is the Fedora Project?"

[Bill Nottingham]

Tom spot Callaway (tcallawa at redhat.com) said: 
> I do however, tend to agree with this user's conclusions: Fedora needs a
> measure of controlled stability and improved usability.
> 
> I think there are a few things that we need to do to accomplish that:
> 
> * Fedora needs a dedicated focus on usability. This is something that
> requires coordinating the efforts of designers and programmers, along
> with usability testing. I'm proud to have been able to take the first
> baby step towards that by providing Mo with a portable Usability lab, so
> we can begin gathering data and doing research, but there is much that
> still needs to be done.

I'd agree here. I think I'm like many people here - I don't necessarily
have the ability to make good design, but I can recognize bad ones. A
non-trivial amount of software (that we ship, even!) falls into the bad
category, and it's because there wasn't much thought given to the design
when it was created (whether it's user interface, admin interface, or
even API design.)

> * Fedora needs a dedicated focus on QA. This is something where I feel
> confident we are currently making solid progress, especially around
> AutoQA, but we are not making enough noise about. The fact that Chris
> Aillon (a Board member) was unaware of this initiative illustrates that
> failing. :) Our improved Test Days and Bug Triage are wins, but we need
> to continue to be more aggressive here, and try to find ways to involve
> and incorporate our community.
> 
> * Fedora needs to improve how it handles updates. Part of this problem
> is defining what merits an update. Some of this is covered by the
> Critical Path initiative, but I think we can build upon that foundation.

These two in combination I think are the biggest win we can possibly
have. As long as updates seem to follow an 'untested, anything goes'
mentality, I don't see how we can target *any* use case beyond masochists.

> Just off the top of my head, how about something like this:

As a note... this may be better targeted at FESCo. Not sure.

> * Clearly mark Critical Path packages as such in Fedora infrastructure
> * Critical Path packages may not do "enhancement" updates on a
> non-rawhide release branch (exceptions permitted only with FESCo approval).

This may be a bit stringent. To grab an example, we push updates for
yum to previous releases. It includes bugfixes, but also changes like:

commit daab5bd8a7bf33902789e4f032babb2e88ac2f0d
Author: James Antill <james at and.org>
Date:   Mon Aug 17 11:28:33 2009 -0400

     Add optimized simplePkgList() to sqlitesack, significant speedup:
    
which could be considered enhancements.

> * Critical Path packages must have a QA test plan for updates to ensure
> that there is no loss in functionality.

There's a chicken/egg issue here - will we not allow security updates
until the maintainer writes a test plan?  We may have to be lenient.

> * Where applicable, the user experience should not change for a Critical
> Path package as part of an update (with the notable exception of a bug
> fix or security hole closed)
> * Packages not defined as Critical Path are permitted to do
> "enhancement" updates on a non-rawhide release branch, but are strongly
> encouraged to minimize the amount and frequency of these updates.
> * Any non-Critical Path update which alters the user experience must be
> documented as a part of the update announcement, and announced to the
> relevant mailing lists (perhaps all "enhancement" updates go out to
> fedora-list?)

This all sounds good to me. Of course, there's the issue of packages
where the security updates come via updating to the latest upstreem.

> FWIW, I also think that "updates-testing", as it is today, does not work
> for Fedora. In all of my packages, I am lucky if I can convince even one
> individual to provide karma on an update, and I have never managed more
> than that, even when I know there are tens/hundreds of users aware of
> the bug (and waiting for the update to fix it). A few ideas on how to
> fix it:
> 
> * Make a period of time in updates-testing mandatory for all updates.
> This can still be overridden by "bodhi karma" votes from testers, but
> nothing can be pushed directly to stable. I'm not a fan of this on its
> own, as I think it will merely encourage people to game the system, as
> we have seen before when individual maintainers have imposed similar
> policies on their own packages... but if paired with my other idea...
> 
> * Encourage community testing of updates-testing, via "Fedora kudos". If
> every package had a list of functionalities and features, and
> instructions on how to test those features, every update would be
> reasonably testable by a competant Fedora user. Any user who tested an
> update and indicated that it:
>  - No longer illustrated the bug it fixed.
>  - Functioned as expected and documented
> Would receive "a Fedora kudo". (Heck, they'd even get one if they showed
> that the update was broken, that's just as useful to know!)
> We'd also give out kudos for users who help define the
> functionalities/features of a Fedora package (with screenshots, testing
> commands to run). Package maintainers can always sanity check these, and
> we will also want to encourage folks to be doing peer review of such items.
> 
> This requires some infrastructure to be built to enable this, but I
> think the payoff potential here is huge. I'm hopeful that we can do this
> as part of the next major milestone of the "Fedora Community" Moksha
> project.

That's a neat idea. To expand on it, I think we need to make it easier to
provide this feedback for users? Ideally, we should be able tell on a
user's system what packages they have that are test updates, have them
easily provide such feedback, and automatically know when those updates
go final (so we don't prompt them about something that's already final.)

Bill