how to improve the efficiency [Re: Round-up, 2004-11-28]

[Pekka Savola]

On Fri, 3 Dec 2004, Eric Rostetter wrote:
>> Use of PGP signatures:
>>   - Is it necessary to use PGP signatures when reporting at bugzilla?
>
> YES!
>
>> Bugzilla already provides user authentication, so this only gives
>> relatively little additional protection.  It's much simpler if you
>> would not need to hassle with PGP at all if you just want to report
>> whether a package works or not.  It's (more) OK to require the use of
>> PGP for those who submit the actual packages, etc., though.
>
>
> See the security page.  Digital signatures are critical to the project.
> No other way can one establish trust, and the project depends on a trust
> model to succeed.

At least http://www.fedoralegacy.org/about/security.php does not 
describe this at all.

Note that I'm not arguing that the RPMs should not be signed, but that 
the reviewers would not have to sign their comments if they didn't 
want to bother.  There is no gain in requiring that.

If you want to get the past track record, you can just search for the 
bugzilla account, not for the PGP signature.

>>   - For example: updates-testing says:
>>
>>     1. Download the binary RPM package from the updates-testing channel.
>>     2. Verify the integrity of the downloaded package (see
>> http://www.fedoralegacy.org/about/security.php).
>>     3. Install the package, and note any installation problems.
>>     4. Use the package (as appropriate for the package), and note any
>> problems found.
>>
>> These must be more explicit.  What exactly must be verified at "2"?
>
> Well, you refer to the link given, which tells you how and what to verify.

So, you'll only check the PGP signature?  That's simple, sure. If 
there is nothing more to do than to check that the PGP signature is OK 
and it installs nicely, why don't just say that?

E.g.,

Verify the PGP signature of the downloaded package 
(see the details at http://www.fedoralegacy.org/about/security.php).

As it is, the statement seems to imply there might be a lot more to 
verify than just PGP signature of the RPM.

>> How do you report being successful after "4"?
>
> You see the next section of the page, IIRC.

Not sufficiently concise what must be done (an example might not 
hurt, e.g., a pointer to a bug and referring to the comment number to 
look at).

>> The same applies to the rest.
>
> I don't see your point.  I think you simply failed to follow the links,
> or read the rest of the page.  But that doesn't matter.  It is a wiki
> page.  You are free to modify it to make it better.  You are free
> to suggest additions and changes to it.  Please participate!

No, my point is that it _might_ (or not, not sure) be sufficient to 
get a feeling what you should do if you scanned the whole website, and 
read it intensively for hours.

That's not good marketing. :)

Unless the website can describe in a manner that can be understood in 
less than 5 minutes how you can help in the process, it doesn't help 
those people (most of them :) who don't have hours to devote to 
finding all the details -- at least at first.

The learning curve must be _very_ low.  That's only way to get 
participation.

> There is no such thing.  First, above is a 4 step list.  You don't like
> it.  So it won't work.  Second, each package is different.  How you verify
> a package depends on the package.  Different packages need different
> verification processes.  Third, each tester's machine is different.  One
> may install with YUM, one with APT, one with RPM of the binary RPM, one
> by rebuilding the source SRPM, etc.  Some will be missing dependencies.
> Others will have conflicting packages, or newer versions installed, etc.
> Some will use lilo, others grub, etc.  We can't provide a simple three
> step process that addresses all the possible situations and variations.

The issues you've pointed out are things that any averagely clued 
administrator familiar with Red Hat Linux should be able to figure 
out.  We don't need to give out the exact commands how to fetch the 
package, or which apt/yum/etc. parameters to use to install them.  We 
can assume that the admin can figure out if we say 'install the 
package, from URL:xxx'.

But we'll need _explicit_ instructions (IMHO) for the fedora 
legacy-specific stuff, which we cannot assume the admin is familiar 
with.

>> If it's a lot of steps (more than, say, 5),
>> we'll want to create scripts to help in the process -- e.g., one which
>> compares the original SRPM and the updated SRPM and shows the results
>> (recursing through tarballs etc.); one that does the same for
>> binaries; etc. -- I think these have already circulated on a list half
>> a year ago or so.
>
> Yes.  I wish the people with these scripts would put them up in the wiki
> with instructions on how to use them, etc.  So far, no one has taken up
> my challange to do that, AFAIK.

Yeah.. this is a BIG problem for doing meaningful upgrade testing. 
(Not so much at updates-testing phase, but in the previous stage).

>> There should also be more documentation at least on following:
>>   1) documenting new vulnerabilities (I think Jesse has mostly been
>> doing this) -- in bugzilla?
>
> What isn't covered about this?  If you tell me what is missing, I'll
> add it.  If the problem is you don't know where to look, let me know
> and we'll try to make some cross-links to make things easier to find,
> or menu changes to make it easier.  But you need to tell us what the
> problem is.

What kind of information do you need to put in the report?  How do you 
articulate the subject, what other special keywords you put there, 
etc.?

> As for docs, I've asked many, many times for people to participate.  Only
> about 3 people have ever taken up the call.   We need more paritipation.
> We need more discussion, more ideas.  Please help!
>
> Yes, I'm being rather obtuse about the documentation changes recommended
> above, but it is mostly due to the lack of content in the suggestions.
> Please address each point separately, and we'll hash them out and get
> them resolved as best as possible.  I'm not against changes or better
> docs.  I just have only so much time to devote to it, and so much talent
> to go into it, and only one perspective when looking at the docs.  I need
> help to improve them, and the only way, and best way, to get that help
> is via the participation of as many people as possible in discussion about
> the current docs or lack thereof.

There seem to be two options:
  1) someone really familiar with an optimal process jumps up to 
describe/organize it better for the FL QA beginners, or
  2) a FL QA beginner with a lot of free time comes up, figures out 
everything on his own, and then suggests fixes to the documentation. 
This would take a LOT of time (I'd say in the order of week), so I 
fear this isn't going to happen.

I'd really hope that someone from category 1) could try to re-think 
the documentation from a FL tester's perspective -- and when it would 
be much better, later the the huge influx of FL testers (yeah, right 
;-) could also step up and suggest minor enhancements.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings