Thoughts on Eric Raymond's Insights

ByteEnable ByteEnable at austin.rr.com
Tue Mar 2 21:04:23 UTC 2004


On Tue, 2004-03-02 at 13:44, Jef Spaleta wrote:
> Jonathan M. Gardner wrote:
> > I think QA and usability tests are two different things
> 
> I'm not saying they are the same, I'm just pointing out
> there is a lot of work to go around for potential volunteers.
> I'll take whatever opportunities to recruit for the community based
> efforts I care about...even if its at the expense of someone else's
> idea...I'm evil like that. Everything is competent manpower limited.
> And I plan to do what I can to make sure my pet projects take
> as much of that limited resource as possible.... better hope you
> convince me your idea is something i should add to my list of pet
> projects.
> 
> > I see your point. I'll do an experiment at home. I have a lovely 
> > wife who hates computers but uses them anyway. 
> 
> I would call that a bias. And from my point of view, the results of the
> session with your wife aren't as important as the methodology
> you use to pick the test-subject(your wife) and the methodology you use
> to identify the usability issues you want to get feedback on, so that
> your session can be repeated by others. If its not really
> repeatable...the session with your wife is going to be nothing better
> than a random osnews review, where personal preferences get expressed
> without a meaningful framework of comparison. And we all know how much
> traction those sorts of reviews in the media have on developer
> thinking....
> 
> You are going to have to be extremely careful that you approach things
> in a way that only addresses usability and not utility.
> And actually, i would argue that you should attempt to address just
> a single aspect of usability at a time. I really think this sums up
> things in very broad strokes:
> http://www.useit.com/alertbox/20030825.html
> 
> But all the links here are of value:
> http://fedora.redhat.com/participate/developers-guide/s1-ui-more-suggestions.html
> particularly for you maybe this:
> http://usability.gov/methods/usability_testing.html
> 
> > Are there any volunteers for a project that would be willing to 
> > take the results of a usability study seriously?
> 
> Depends on how serious the usability study is....
> And I doubt all the project developers actively read all the messages in
> this list. You are going to have to be proactive and try to poke a few
> developers in the eye who are leading projects that could use some help
> with usability. Since the gnome people already have a usability project
> listed...
> http://developer.gnome.org/projects/gup/references.html
> you might find a person associated with that who is willing to provide
> some guidance.  KMail and Mozilla Mail and Evolution are all
> upstream development issues, not under the direct control of Fedora
> Project. You will have to start getting involved in the upstream
> mailinglist and approach the developers on their home turf. 
> redhat-config-network or as we like to call it now system-config-network
> is the providence of:
> http://fedora.redhat.com/projects/config-tools/
> and has its own mailinglist. It's up to you to find out which project
> would be most receptive to your experimentation.
> 
> 
> > But in my experience, even though they don't turn out a great 
> > volume of statistics with error bars or complete analysis of 
> > the target market, they do give a lot of valuable feedback.
> 
> No offense, but for the sake of this discussion, its not clear how
> valuable your personal experience is on this matter. And to be fair
> its not clear how valuable my personal experience on this matter is
> either. And in my experience, just saying 'in my experience' without
> some noteworthy credentials or a working relationship with your audience
> based on a good track-record of productive conversation doesn't help
> convince people who are leaning towards a healthy level of criticism
> when listening to your opinions. Its far better to cite references that
> look authoritative. Even if they are crap, it at least shows you aren't
> just talking only from a point of view of personal opinion and are
> willing to do some research.  
>  
> -jef"looking forward to seeing your methodology"spaleta
> 

On note on usability.  I agree with Jef on some of his comments, in
particular that one person is not enough data to justify an entire
product change.  In the industry this is called a focus group.  Let's
say, that you identified five areas (top five) that all users had
problems with using your software.  Obviously, you would address those
top five issues.  However, those users in the focus group should be
selected based on the market or group of users that the product is
targeted for.  For a very broad classification, one could draw some
ideas from this list:  

o  beginners - total computer noob
o  intermediate - some computer experience
o  advanced - Master of more than one application
o  crossover1 - moving to Linux from Windows 
o  crossover2 - moving to Linux from Mac
o  crossover3 - you get the idea

Now, someone with a human factors PHD needs to come up with a human
factors test that someone could administer without having to have a PHD
in human factors.  Ideally, it would be a software program that would
save the results for later upload to a database.  Or you could have a
Internet Human Factors site where you could administer the test from so
the data is collected real time.  Sounds like a good graduate project
somewhere.

Byte





More information about the fedora-devel-list mailing list