[libvirt] Scope of the test driver? (Was: Re: [PATCH] test_driver: implement virDomainSendKey)

[Erik Skultety]

On Mon, Jun 03, 2019 at 12:31:56PM +0200, Peter Krempa wrote:
> On Mon, Jun 03, 2019 at 08:52:55 +0200, Erik Skultety wrote:
> > On Mon, Jun 03, 2019 at 08:40:13AM +0200, Pavel Hrdina wrote:
> > > On Mon, Jun 03, 2019 at 08:23:57AM +0200, Erik Skultety wrote:
> > > > On Sat, Jun 01, 2019 at 02:46:56PM +0200, Ilias Stamatis wrote:
>
> [...]
>
> > > > I think this API should be merely about translating the value, because
> > > > this way you're just blocking a synchronous API for no apparent benefit. I
> > > > believe it should return instantaneously. On the other hand, I'm just wondering
> > > > what value it brings having an API translating keycodes, but I guess for
> > > > consistency reasons, we can happily introduce it.
> > >
> > > The benefit is to simulate the holdtime as for example hyperv driver is
> > > doing, in QEMU we just pass everything to the QEMU itself where that one
> > > will probably do similar operation.
> >
> > Both hyperv and vbox have to do it because they don't support it, so they do
> > usleep before sending down and up keycodes respectively, QEMU does it
> > iternally. In test driver, you're not testing any true codes, so doing plain
> > usleep si IMHO wrong, from app dev's perspective you're only testing whether
> > usleep works by providing arbitrary time to wait, which raises the obvious
> > question "why".
>
> I think the problem here is that it's unclear what the purpose of the
> test driver is supposed to be. Clarifying the purpose first might have
> positive impact on the design decisions here.
>
> So what's the point of the test driver?
>
> Is it meant for human interaction, e.g. for application developers or
> users wanting to test stuff? In that case you probably want to simulate
> the "worst" behaviour we have, which would be the blocking timeout so
> that users/devs can see the worst case implications of running such a
> command.

I agree and disagree at the same time. The part I disagree with is that anyone
would be truly interested in seeing how much the API blocks, they'd just want
to job done and as such the wait doesn't bring any value to that and I agree
I'd also recommend doing anything related to ^this to be done on dummy VMs as
you mentioned below, but then again, we're focusing on test driver coverage, so
from coverage perspective, we probably want to cover this API too - to some
degree.

Thanks,
Erik

>
> Alternatively the purpose may be to allow some kind of "unit" testing
> for APPs using libvirt. (I specifically used "unit" testing, because any
> kind of sane real integration testing will not use the test driver
> because it would not accomplish much). In such case the timeout is a
> hassle because it just unnecessarily prolongs test runs and does not add
> much value.
>
> Any other ideas? I don't have any more.
>
> Frankly even the two use cases above aren't something I'd recommend to
> the users. In case of user/dev testing, checking against a real VM makes
> more sense and the same goes for real integration testing.



> --
> libvir-list mailing list
> libvir-list at redhat.com
> https://www.redhat.com/mailman/listinfo/libvir-list