QA Application XML save formats.

[Erik LaBianca]

> -----Original Message-----
> From: fedora-devel-list-bounces at redhat.com [mailto:fedora-devel-list-
> bounces at redhat.com] On Behalf Of Toshio
> Sent: Sunday, May 30, 2004 11:29 AM
> To: fedora-devel-list at redhat.com
> Subject: QA Application XML save formats.
> 
> Hi,
> 
> I've got a first cut of an xml save format for my QA Assistant App.  If
> Aurélien, Erik, and others interested in tools that help write QA
> reports want to take a look, I'd be glad of any input.  I'm only a
> novice xml author so my format could probably use some work.
> 

Sorry I haven't been following your qa-assistant very closely, I don't run X most of the time. I did download the rpm from sourceforge, and took a look at it. I'm assuming that version doesn't know how to save the .xml files you referred to?

Since you have taken the time to encapsulate the checklist as an XML file, I think we should all start adopting THAT as the canonical reference, including going so far as to write an xslt sheet to render it as html and posting it on the website.

I've created a first round xslt stylesheet, which can see the results of at
http://www.ilsw.com/~erik/fedoraus.xml

The xslt stylesheet is at 
http://www.ilsw.com/~erik/checklist.xsl

I also spent some time thinking about how to integrate our code. I'm going to throw out an idea here, and you can see what you think. If you look at the changes I made to the first couple of entries in fedoraus.xml, you can see I added a <script> section, with some simple code in it. My thought is that the best way to enforce a policy is to put the code to do it in the same place as the policy definition, aka the checklist definition. 

I also created a simple python script to walk through the checklist, find the scripts, and run them. With some tweaking, I think it could work admirably both for an unattended checker, or integrated into a gui like yours. The script is at http://www.ilsw.com/~erik/runchecklist.py

Outstanding issues:
We need to agree on a way to define input parameters to pass into scripts, ie things like a gpg keyring file, or the srpm filename, or ??? You can see I started doing this, but I haven't finished the job by any means.

We need to agree on an output method. I think the easy way to do it is by mapping return codes to states like I show in the example, or taking the entire textual output of the program and matching it to a state name. We can do both, they're both easy.

Other comments:
I'm not sure I entirely understand the difference between qasave.dtd and checklist.dtd. My personal preference is to generate the qasave dtd (xml schema would be better) and xml directly from the checklist xml. Then there is 1 and only 1 definition of the policy, description, and storage format.

I think the checklist probably needs to carry more of the policy information we've argued about (and not decided on) with it. IE I'd like for each entry to have a "class" or something indicating if it was a Blocker, Non-Blocker, Important, or Cosmetic only.

I think the checklist xml also needs to carry whatever information might be needed to create a QA report in the affirmative or negative for that particular entry. IE you should be able to create some sort of human friendly listing of the packages status by transforming a "checklist data file" and a "checklist" together. Maybe you already have this?

Anyway, I don't have more time to work on this today, I thought I'd get my thoughts out there and see what you all think

--erik

p.s. It's been a few weeks since the redhat conference, and I haven't seen a single status update about fedora extras. Am I blind? 

I hate the idea of spending a lot of time putting together QA tools and then having the policy and methodology utterly pre-empted by decree when redhat releases their external contributor policy and infrastructure.