[libvirt] [PATCH] docs: document that C & Python are the preferred languages

[Daniel P. Berrangé]

On Thu, Sep 05, 2019 at 06:15:04PM +0100, Daniel P. Berrangé wrote:
> On Thu, Sep 05, 2019 at 12:30:27PM -0400, Laine Stump wrote:
> > (BTW, what does the removal of perl from libvirt say about continued
> > use of perl for libvirt-tck? There are a lot of useful tests in there
> > that find real bugs, but they tend to languish (both in use and in
> > enhancements) because nobody wants to do anything with perl (or with
> > the big shell scripts that do the nwfilter testing). It would be nice
> > if the tests were all in a language that was more accessible, but
> > they're kind of married to the perl TAP module (and besides, who
> > wants to spend time rewriting a bunch of test scripts when they
> > already work?). This is mostly a moot point, because I think hardly
> > anyone runs the libvirt-tck tests anymore, which is too bad because
> > it has historically caught some regressions that no other testing
> > framework did.)
> 
> I believe they are run by the Red Hat virt QE team, as I ave
> got bug reports against the Perl libvirt bindings, where the
> reproducer is a TCK script.
> 
> The TCK really covered three distinct things
> 
>  - framework for controlling test execution
>  - library modules/helper APIs to facilitate test creation
>  - the large set of indivdual tests
> 
> There is no code binding between the tests and the framework. They
> communicate exclusively through the TAP protocol. In that way we can
> throw out the current framework and drop in a different one quite
> easily - the new framework merely needs to be able to consume the
> TAP data format. For any framework which does not natively support
> the TAP format, it is quite easy to write an adapter / shim that
> would convert to the desired data format. You might loose some
> finese / fine grained details in the logs/progress reporting but
> functionally the tests should work fine.
> 
> As a point of reference the GLib testing harness has recently
> switched from using their own custom output format to using the
> TAP output format. So if we adopt GLib in the libvirt.git and
> use it in our unit tests those would end up using TAP. GLib
> provides a shim for automake that allows it to consume the
> results from GLib based units tests.
> 
> QEMU's makefiles recently switched to using TAP for its unit
> and (some of) its functional tests too.
> 
> IOW, I'm perfectly happy to throw away the existing TCK framework
> impl if someone proposes a better alternative.

I came across this the other day, which looks like it could be a
viable replacement for the TCK control framework:

  https://github.com/python-tap/tappy
  https://tappy.readthedocs.io/en/latest/


> 
> 
> As for the individual TCK tests themselves, there's two distinct
> questions
> 
>  - Should they be re-written in a different language
>  - Should they continue to use TAP output format
> 
> If we are not going to rewrite the tests, then I think they should
> continue to use TAP output format. It is the easiest thing to use
> when writing tests in Perl.
> 
> If we are going to rewrite the tests, then the tests should ideally
> use the output format that best suits the framework invoking them.
> IOW, if the framework still uses TAP, we should continue to use
> TAP. If not, then don't. There is an adapter for python that can
> convert its native unittest output format into TAP format.
> 
> 
> I'm pretty ambivalent on rewriting the existing tests. The language
> of existing tests only matters if you expect people to be modifying
> them. Assuming a decent support library it should be no harder to
> define a new test, than to extend an existing test.
> 
> IOW, if we want to write TCK tests in Python, we don't really need
> to rewrite existing ones. We instead need to write a Python variant
> of the helper APIs, so that you can trivially write new Python tests
> which operate in the same manner as the Perl tests.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|