[libvirt] Supporting hypervisor specific APIs in libvirt

Tue Mar 23 11:33:57 UTC 2010

On Mon, Mar 22, 2010 at 04:49:21PM -0500, Anthony Liguori wrote:
> On 03/22/2010 03:10 PM, Daniel P. Berrange wrote:
> >>This isn't necessarily libvirt's problem if it's mission is to provide a
> >>common hypervisor API that covers the most commonly used features.
> >>     
> >That is more or less our current mission. If this mission leads to QEMU
> >creating a non-libvirt based API&  telling people to use that instead,
> >then I'd say libvirt's mission needs to change to avoid that scenario !
> >I strongly believe that libvirt's strategy is good for application
> >developers over the medium to long term. We need to figure out how to
> >get rid of the short term pain from the feature timelag, rather than
> >inventing a new library API for them to use.
> >   
> 
> Well that's certainly a good thing :-)
> 
> >>However, for qemu, we need an API that covers all of our features that
> >>people can develop against.  The ultimate question we need to figure out
> >>is, should we encourage our users to always use libvirt or should we
> >>build our own API for people (and libvirt) to consume.
> >>
> >>I don't think it's necessarily a big technical challenge for libvirt to
> >>support qemu more completely.  I think it amounts to introducing a
> >>series of virQemuXXXX APIs that implement qemu specific functions.  Over
> >>time, qemu specific APIs can be deprecated in favour of more generic
> >>virDomain APIs.
> >>     
> >Stepping back a bit first, there are the two core areas in which people can
> >be limited by libvirt currently.
> >
> >  1. Monitor commands
> >  2. Command line flags
> >
> >Ultimately, IIUC, you are suggesting we need to allow arbitrary passthrough
> >for both of these in libvirt.
> >
> >At the libvirt level, we have 3 core requirements
> >
> >  1. The XML format is extend only (new elements allowed, or add attributes
> >     or children to existing elements)
> >  2. The C library API is append only (new symbols only)
> >  3. The RPC wire protocol is append only (maps 1-1 to the C API generally)
> >   
> 
> We have a slightly different mentality within QEMU I think.  Here's 
> roughly how I'd characterize our guarantees.
> 
> 1. For any two versions of QEMU, we try to guarantee that the same VM, 
> as far as the guest sees it, can be created.
> 2. We tend to avoid changing command line syntax unless the syntax was 
> previously undefined.
> 3. QMP supports enumeration and feature negotiation.  This enables a 
> client to discover which functions are supported.
> 4. We try to maintain monitor interfaces but provide no guarantees of 
> compatibility.
> 
> >The core question for us as libvirt developers is how we could support
> >QEMU specific features that may change arbitrarily, without it impacting
> >on our ability to maintain these 3 requirements for the non-hypervisor
> >specific APIs.
> >
> >We don't ever want to be in a situation where a QEMU specific API will
> >require us to change the soname of the main libvirt library, or introduce
> >incompatible wire protocol changes. If we were to introduce QEMU specific
> >APIs, we also need a way to easily remove those over time, as&  when we
> >have them available as generic APIs.
> >
> >At the C API level, this to me suggests that we'd want to introduce a
> >separate libvirt-qemu.so  library for the QEMU specific APIs. This
> >library would not have the same requirements of fixed long term ABI
> >that the main libvirt.so did. We'd add QEMU APIs to libvirt-qemu.so
> >any time needed, but remove them when the equivalent functionality
> >were in libvirt.so, and increment the soname of libvirt-qemu.so at
> >that point.
> >   
> 
> How different is having a libvirt-qemu.so from having a libqemu.so that 
> libvirt.so uses?
> 
> Practically speaking, if libvirt-qemu.so uses a separate XML namespace, 
> does the fact that we use a different config format matter since you can 
> transform our config format to XML and vice versa?

If an application used libqemu.so directly, they would be excluding themselves
from much of the libvirt ecosystem which would otherwise be beneficial to their
needs. For example, with libvirt-qemu.so, you could still use libvirt's secure
remote API access, all the authentication & authorization capabilities on the
API. It could also be intergrated into the other libvirt language bindings, and
mapping layers such as libvirt-CIM, libvirt-qpid, etc. Applications may still
also want the benefit of the libvirt hypervisor-agnostic APIs, for example, 
virt-manager wants to use libvirt.so primary so it can support QEMU, Xen, VMWare
etc, but it might also want to access qemu specific features via libvirt-qemu.so
It could not use libqemu.so because it would not be accessible via the libvirt
remote RPC layer.

> I think the problem is, if libvirt.so introduces a common API, removing 
> it from libvirt-qemu.so is burdensome to an end-user.  For someone 
> designing a QEMU specific management application, why should they have 
> to update their implementation to a common API that they'll never use?

I think we'd have to have a formal deprecation cycle, to allow overlap
between adding a feature to libvirt.so & removing it from libvirt-qemu.so
Even if they are building a QEMU specific mgmt application, there is still
value in switching to a libvirt.so based API, because with that we are in
a stronger position to guarentee their app will continue working with future
QEMU that may change its command line or monitor syntax. 

For example, consider if a libvirt already had support for passing qemu
specific args through to QEMU, and that app were using '-net' for some
host style networking option libvirt.so didn't initially support. Now 
QEMU introduced '-netdev' as an improved option for networking. If the
QEMU specific mgmt app had switched to the generic libvirt API once it
was available, they would have got the benefit of the new improved -netdev
option automatically. This is even more compelling in cases where QEMU has
actually removed the existing argument.

> >>What's the feeling about this from the libvirt side of things?  Is there
> >>interest in support hypervisor specific interfaces should we be looking
> >>to provide our own management interface for libvirt to consume?
> >>     
> >Adding yet another library in the stack isn't really going to solve the
> >problem from the POV of libvirt users, but rather fork the community
> >effort which I imagine we'd all rather avoid. Having to tell people to
> >switch to a different library API to get access to a specific feature
> >is a short term win, but with a significant long term cost/burden. This
> >means we really do need to figure out how to better/fully support QEMU's
> >features in libvirt, removing the feature timelag pain.
> >   
> 
> I think what we need to do is find a way to more tightly integrate the 
> QEMU and libvirt communities in such a way that when a patch was 
> submitted against QEMU adding a new feature, we could ask that that 
> feature was implemented in libvirt.  I see two ways to do this.
> 
> One would be for libvirt to have a libvirt.so and libvirt-qemu.so.  The 
> QEMU community would have to be much more heavily involved in 
> libvirt-qemu.so and it probably suggests that libvirt-qemu.so should 
> follow our release cycle.  libvirt would have to support using either 
> libvirt.so or libvirt-qemu.so for it's users.

I'm not anticipating an either/or scenario for this. Apps would primarily
still use libvirt.so for everything. They would only use libvirt-qemu.so
to access indiviaul QEMU specific APIs they might need. In other words, it
augments the functionality of the base libvirt library.

> The alternative would be for the QEMU community to produce a libqemu.so 
> and for libvirt.so to consume libqemu.so.  The libvirt community ought 
> to be heavily engaged in the development of libqemu.so and certainly, 
> shared maintainership would be appropriate.  A user using libvirt.so 
> should see guests created with either libqemu.so or libvirt.so although 
> libqemu.so would provide weaker long term compatibility guarantees (but 
> more features).
> 
> I think both approaches are technically workable.  The question is which 
> is the most appealing to both communities and which requires the least 
> development effort.

I don't neccessarily think that the approaches are mutually exclusive
either, because in some ways they are addressing different problem 
spaces. Even if there was a libqemu.so that libvirt used, there would
still be demand for a libvirt-qemu.so, because apps using libvirt for
mgmt do not want to discard use of the entire libvirt API just to access
1 extra QEMU specific feature. A libqemu.so doesn't solve that problem,
its just an invisible internal impl detail for libvirt as far as end
users are concerned.

> At a high level, I think libqemu.so would require a heavy refactoring of 
> the qemu libvirt driver.  Guest enumeration would have to be handled by 
> qemu and libvirt would need to support translating qemu's config format 
> to an XML format.  It would also need to translate the qemu XML format 
> into it's own XML format relying on an xmlns to support the features 
> that are not currently supported by the common hypervisor API.

Both of those are minor implementation details ;-)

> The advantage though would be that any qemu instance launched would be 
> visible to libvirt and libvirt tooling.
> 
> libvirt-qemu.so would require less change to the qemu driver in libvirt 
> but would require some awkwardness in terms of translating a guest 
> config to an XML config.  Some of the concepts that qemu supports (like 
> running a guest as non-privileged user using stdio) would be difficult 
> to support.

Translating from libvirt XML to a QEMU specific config format is really
not a burden. This is a core task for any libvirt hypervisor driver. It
just happens that the current "config format" is a list of ARGV. Within
reason, any other config format is just as managable. It only becomes
hard when a hypervisor's config format arbitrarily changes semantics with
each new release :-(

Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|