[libvirt PATCH 9/9] tests: Add tests for unknown elements and attributes in cpu defintion

[Tim Wiederhake]

On Wed, 2020-09-30 at 14:11 +0200, Peter Krempa wrote:
> On Wed, Sep 30, 2020 at 13:54:58 +0200, Tim Wiederhake wrote:
> > On Tue, 2020-09-29 at 11:22 +0200, Peter Krempa wrote:
> > > On Mon, Sep 21, 2020 at 15:07:32 +0200, Tim Wiederhake wrote:
> > > > Signed-off-by: Tim Wiederhake <twiederh at redhat.com>
> > > > ---
> > > 
> > > [...]
> > > 
> > > > diff --git a/tests/cputestdata/x86_64-bogus-element.xml
> > > > b/tests/cputestdata/x86_64-bogus-element.xml
> > > > new file mode 100644
> > > > index 0000000000..79f98bad18
> > > > --- /dev/null
> > > > +++ b/tests/cputestdata/x86_64-bogus-element.xml
> > > > @@ -0,0 +1,3 @@
> > > > +<cpu>
> > > > +  <nonExistantElement/>
> > > > +</cpu>
> > > 
> > > I'm not persuaded that there's value in such test.
> > > 
> > 
> > This was my approach to adding negative tests to the "virsh
> > [hypervisor-]cpu-compare --validate" feature to make sure that
> > schema
> > violations are actually caught. If this is not the correct place to
> > do
> > so, I would be grateful for a pointer in the right direction.
> 
> I'd find way more useful to actually test that all the XMLs we use in
> the tests are validated, which does not seem to happen in this
> series.
> 
> This series adds schema validation only for a very synthetic negative
> case which IMO doesn't make sense.
> 
> Additionally we do have 'virschematest', which is meant to validate
> all
> XML documents in the tests. Based on the fact that you've added a
> invalid XML example but it's name doesn't end in '-invalid.xml',
> which
> is the marker for the 'virschematest' worker to do negative
> validation,
> means that the cpu files are not validated.
> 
> Rather than adding a synthetic case, wire up virschematest which will
> make sure that we don't have bogus XMLs in the test suite.

Thanks, I was unaware of this test. One problem I see is that
tests/cputestdata contains xml files with different root elements:

$ for i in tests/cputestdata/*.xml ; do xpath -q -e "name(/*)" $i ;
done | sort | uniq -c
    273 cpu
    157 cpudata
     16 cpuTest

Adding 'DO_TEST_DIR("cpu.rng", "cputestdata");' to virschematest.c
therefore fails for 157 + 16 files. I do not believe that we would want
to individually list 273 files in virschematest.c and reworking the
directory structure appears undesirable. Do you have any suggestions on
how to go forward on this?

Thanks,
Tim