"Features to test" list and documentation on proper testing and diagnosis of issues

Jef Spaleta jspaleta at princeton.edu
Fri Apr 9 20:12:51 UTC 2004


Brian Bober wrote:

> I think it'd be good if developers described what was changed, 
> enabled, removed
> in every test release and major package (i.e. kernel) so that when guys like me
> are bored, we can just sit there fiddling around with what was changed to see
> if we find any problems. You mentioned testing SELinux, but there are other
> things you could ask us to test. Also what would be useful information on how
> to diagnose problems and how to test it.

Sadly, creating usable documentation aimed at better guiding testers, is a manpower
issue as much as anything else. The developers are having a hard enough
time just keeping up with technical issues as they are reported, so even
if you could sucker them into writing guidelines aimed at inexperienced
testers, it would be difficult for it to bubble up to a reasonable
priority on a list of things to do.

We have to bring some extra manpower online to sketch out the issue of
documentation and guidance to testers for it to get anywhere. I don't
know if the fedora-docs project is really the best place for this effort
of if a new effort specific to streamlining communication during testing
phases is in order. I wish i had more time to actually sit down and work
on this issue myself. 

Here is where my head is at about testing guidance in general:
- there is a need for a general introduction document, sort of a
manifesto laying out the intent of testing and the expectations testers
should come into the process with. A little hand hold about bugzilla
perhaps as well.  Very static document i would imagine. 
- more directed documentation about common approaches to gathering
information and preparing a bug report. Not so fined grained as being
package by package.  This i expect would be a living document, with
additions pretty much all the time.
- communication of time based targets or focus during testing phase.
Maybe a weekly focus.
- maybe a codification of action items like your IEEE1394 example, so
that developers can easily notify testers of specific changes that need
testing. Maybe even have a package by package daily set of action items
testers could look over.  For it to work well there would have to be a
drop dead easy way to build an action item and a drop dead easy way to
sort over action items. The problem here is, if its more work for the
developers, who already don't have enough hours in the day, would
something like this really get used. Making it as easy if not easier for
the developers than the end-user would be important.   

As much as i want people to think about this issue now, i'm of a mind
that its best to wait till after fc2 is out the door to have a focused
discussion on how to make the next testing phase better. I don't want a
'lessons learned' conversation taking cycles away from current testing,
when there aren't enough developer cycles as it is. 

-jef





More information about the fedora-test-list mailing list