[Avocado-devel] Display loader failures

[Lukáš Doktor]

Hello guys,

Cleber asked me on wider feedback regarding the https://trello.com/c/DLxBtb8r/981-display-loader-failures-in-avocado-list-verbose which we supposedly closed on release 51.0 by introducing fake classes for no-tests that are not inherited from `avocado.Test` and are only intended for listing purposes on verbose discovery. We closed the card because this behavior was accepted in optional plugins (and already existed even as run-able tests every-since the beginning in FileLoader), but unfortunately there is one PR that got nacked, because it's not "pure" enough and we mix tests and not-tests in the same return.

My view on this is I don't mind that. Loader returns list of discovered items, in case of verbose discovery they can return objects that are not tests as can be seen in the tests summary:

```
avocado list -V this-file-does-not-exists
Type       Test                                                                                                                                                          Tag(s)
NOT_A_TEST this-file-does-not-exists: File not found ('this-file-does-not-exists'; '/home/medic/Work/Projekty/avocado/avocado/examples/tests/this-file-does-not-exists') 
!GLIB      this-file-does-not-exists: No GLib-like tests found                                                                                                           
!GOLANG    this-file-does-not-exists: Go binary not found.                                                                                                               
!ROBOT     this-file-does-not-exists: Data source does not exist.                                                                                                        

TEST TYPES SUMMARY
==================
!GLIB: 1
!GOLANG: 1
!ROBOT: 1
ACCESS_DENIED: 0
BROKEN_SYMLINK: 0
EXTERNAL: 0
GLIB: 0
GOLANG: 0
INSTRUMENTED: 0
MISSING: 0
NOT_A_TEST: 1
PYUNITTEST: 0
ROBOT: 0
SIMPLE: 0
VT: 0
```

You can see that it clearly reports why this is not a glib, not golang, not robot and not even base-avocado tests, but you have no way of finding why is it not an Avocado-vt test. With the proposed PR it adds:

```
!VT        this-file-does-not-exists: No matching tests                                                                                                                  
```

which can help identifying the issues (it can also report other kinds of issues like misconfigured Avocado-vt etc.) and I think it's very useful. I don't say it's perfect, but I don't see myself spending too much time on this any time soon when we have higher priority tasks like the tests surviving reboots or Job API but if you have some strong opinions, please share them here so we can get support for reporting Avocado-vt loader issues better than modifying the sources.

Kind regards,
Lukáš

PS: The return could be actually improved by stating whether it used `vt-config`, what were the filters, archs, guest-oss, ... but that can be improved later.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20180421/52dd49f3/attachment.sig>