[Avocado-devel] correlating xunit output to job-results

[Lukáš Doktor]

Dne 26. 11. 18 v 16:24 Brian J. Murrell napsal(a):
> On Mon, 2018-11-26 at 13:47 +0100, Lukáš Doktor wrote:
>>
>> Hello Brian,
>>
>> the results and xunit results are independent, but at the time of
>> creating the results we should have all necessary information
>> available.
> 
> Indeed.  I was thinking the same.
> 
>> The question is how to embed the links.
> 
> I don't even care to have actual links.  I just want to see a reference
> to a job-YYYY-MM-DDTHH.MM-UID in the xunit output somewhere.
> 

This is a new information. So are you actually looking for job-YYYY-MM-DDTHH.MM-UID only, or are you looking for test-name => test-output-dir mapping? Because first I understood the second is the main goal but from this note I think you are looking for the job-id and job-dir-name instead.

>> The main problem I see is that the xunit results can be stored
>> somewhere else than in the standard location and (very often) the
>> results are moved as well.
> 
> Indeed, hence my not even wanting links.
> 
>> For that reason absolute path doesn't make much sense to me and
>> relative path could be troublesome as well.
> 
> Agreed.  I'd say forget the links.
> 
>> Do you have a setup where such information would make sense?
> 
> Jenkins.  We import the xunit results into Jenkins which then displays
> them nicely formatted in "Error Message", "Stacktrace", and "Standard
> Output" sections.  We also then add the job-YYYY-MM-DDTHH.MM-UID dirs
> as artifacts.
> 

Sure, that's the common way :-)

>> If not and you are only looking for a way to map these, you can use
>> the serialized test id. The test-reference (and therefor the test
>> results dir) is strictly defined as "$testId-$testName:$testVariant", 
>> therefor when you split the name (or path) by `-` you'll get test-id
>> which is guaranteed to be unique across the job.
> 
> Where/how are any of these referrable?
> 

My idea was to basically do something like:

    for name in xunit.gettests().get('name'):
        tid = name.split('-')[0]
        print(glob.glob("latest/test-results/%s-*" % tid))

which although not nice should always return only 1 match.

> Looking at a given Test instance I can see lots of attributes and
> actually now that I look, I can see several attributes that embed that
> job-YYYY-MM-DDTHH.MM-UID value into pathnames such as _stderr_file,
> _stdout_file, _Test__logfile, _Test__sysinfodir, _Test__logdir,
> _ssh_logfile, _Test__outputdir.
> 
> I can surely peel the value I am looking for out of any one of these
> but of course, that feels like I am peeking under the kimono and using
> something that is subject to change in the future.
> 

The best way is `json` results, which is free-form, avocado machine-readable results. We keep it as stable as possible (I know of one bigger change with the test-id standardization and we kept +1 LTS overlap with backward compatibility).

>> PS: If you know of a nice way to let Jenkins read those files from
>> artifacts instead of embedding them, do let me know.
> 
> I'm not entirely sure what you mean.  Are you looking for Jenkins to
> provide some sort of nice clickable output format for those job-YYYY-
> MM-DDTHH.MM-UID dirs other than just being able to traverse them as a
> "folder" in it's artifacts browser?  If so, I know of no way better.
> 

Yes and I looked for such way when started using jenkins, but at least at that time there was no such way. I mean I can add link to the output, but that does not sound right... Anyway good to know you don't see a better way either.

>> I'd be really interested (as currently I basically store the output
>> twice, which is not really efficient and I have to use `--xunit-max-
>> test-log-chars` to avoid really long outputs...)
> 
> I must not be understanding what you are doing and what you are looking
> for as an alternative.  Maybe if you have a link to an example.
> 

I think you understand well (based on the first response). Basically I run a job, it produces several hundreds of MB of log files, which are attached as artifacts and then several tenths of MB is attached again as the XUNIT file, because the "debug.log" is always embedded into the XUNIT results. It'd be nice to simply add link to the archived results, but I haven't found a way back then...

Regards,
Lukáš

> Cheers,
> b.
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20181126/96499523/attachment.sig>