New look and feel discussions

[duffy at dangerouslyinc.com]

Hiiii!

>  * How are we going to progress with this?
>
> There was a mention of usability testing. Is this our first port of
> call, or do we want to start with brand new mock-ups? If it's the
> latter, then I'm happy to set up a wiki page with a list of actions
> where we can record how people got on with completing them.

I think usability testing should be the next step. Usability testing is
going to identify what exactly we are doing wrong right now. I think we
mostly know our weaknesses, but we might not understand exactly why the
problems are occurring and I think usability testing will get us there.
Note that usability testing isn't about getting answers on how to fix it,
but we'll certainly be able to identify how exactly the site is causing
user frustration and be able to learn from those mistakes.

When we know what mistakes we've made, we should then work on mocking up
potential solutions to those mistakes.

So how do we do usability testing? Basically, each one of the top ten
tasks is a task on a usability test. We ask folks appropriate for each
tasks to sit down, start at www.fedoraproject.org, and try to complete the
task, speaking aloud (or typing :) ) as much of their stream of
consciousness while they try to complete the task as possible.

So who do we test? It depends on the task. Here are my guesses task-for-task:

1. In one click, download the x86 Fedora Desktop Live Media ISO image. (or
whichever we deem is/would be the most popular / best for new users)

We want anyone from less-experienced folks to experts to be able to do this.

2. Easily access to support tailored for NEW users.

We probably care mostly about brand-new users & less-experienced users on
this one. Experts have other venues for assistance.

3. Figure out what Fedora is / what the project is all about.

We care about folks who haven't become users yet, new / less-experienced
users, and also folks who are evaluating Fedora from a completely
different perspective, such as a member of the news media or a researcher.

4. Join Fedora

Mostly this is probably targeted at new users / less-experienced users,
particularly ones without development/engineering experience as I think a
lot of folks have the perception that you have to be a programmer to
contribute when it couldn't be further from the truth.

5. See / Learn about all of the cool things you can make and do with
Fedora and how other people use Fedora.

I think this one is really targeted at everyone from not a user yet =>
newbie => experienced user.

6. Catch up on what is happening in Fedora, right now? What's the latest?

Current contributors probably care more about this than people who aren't
involved in the community yet.

7. Give back to Fedora. (different from joining. Joining is more of a
commitment)

Newbies/folks without time to become a regular contributor? This task
probably applies to anybody though.

8. Download specific versions / form factors of Fedora. (as opposed to #1
which is one click)

More experienced users / developers should be able to do this; newbies
probably care a little less.

9. Get Fedora bling! Spread Fedora!

Everyone wants this!

10. Access developer tools and documentation for Fedora.

Experienced users/developers mostly.

What do you think about the suggestions above? Based on them, I'd say we
should give volunteers the following tests:

- don't use or know much about Fedora: 1, 9, 3, 2, 5, 4, 6, 7, 8
- new Fedora user: 1, 9, 2, 3, 4, 5, 6, 7, 8
- experienced Fedora user: 1, 9, 2, 4, 5, 6, 7, 8
- Fedora contributor/developer: 1, 10, 9, 8, 6, 5, 7, 8

Once we know what and who to test, well then, we have to test. I'm
thinking best-case scenario we test folks we know in person, less good but
still okay is testing over the phone/ekiga/skype with vnc (because the
person can still talk while completing the task), worst but better than
nothing is over IRC/IM/etc with vnc(kind of hard to type about how you're
doing the task while you're doing it.)

Is anyone here willing to help run tests? If we had five folks willing to
do 2-3 tests each we'd likely have more than enough data to work with.

Would it be useful if I ran a sample test or two and recorded them and
made them available to see? Are there any willing victi^H^H^H^H^H
volunteers that wouldn't mind trying out the 10 tasks and telling me over
ekiga/skype/irc/etc how it goes while they do it and while I observe over
VNC? (is there an easy way to record the VNC session while you watch it?)

~m