[libvirt] RFC: Improving unit test coverage

[Richard W.M. Jones]

> XML parser/formatter reliability
> --------------------------------
> 
> While we have good testing of valid XML documents, there have
> been a number of cases where invalid XML documents have caused
> crashes or other bad behaviour. It is not reasonable to add
> test cases for every possible invalid XML document. Instead
> we need to adopt a fuzzing approach, where we take valid XML
> documents and then make incremental changes to them & then
> ensure that the code does not crash.

For fuzz-testing libguestfs inspection[1] I did this another way,
which might work for XML fuzz testing too.

I modified inspection so that whenever it looked for a file or
directory, the file or directory was created (some random % of the
time) and then inspection is allowed to continue.

For XML what this would mean is you inject code whenever the current
code performs an XPath query or iterates over nodes, which
injects nodes randomly based on the XPath expression or
termination condition of the loop.  As a concrete example, wherever
the code calls:

 virXPathString("string(./monitor[1]/@path)", ctxt)

virXPathString would modify or not (randomly) the XML tree to add a
<monitor path=...> attribute.  Similarly:

 virXPathNodeSet("./vcpus/vcpu", ctxt, &nodes);

can randomly cause <vcpus><vcpu> elements to be added.

The idea anyway is that each possible path through the XML parsing
code is eventually exercised.

Rich.

[1] https://github.com/libguestfs/libguestfs/blob/master/tests/fuzz/fuzz-inspection.pl

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org