[libvirt] libvirt and the lowest common denominator

[Daniel P. Berrange]

On Mon, Feb 09, 2009 at 10:39:55AM -0500, John Levon wrote:
> 
> I wanted to bring this topic out of a patch review thread.
> 
> Dan recently stated that only patches that use "upstream" facilities are
> acceptable, meaning the xend delivered by XenSource. I'd like to make a
> few points on this note:
> 
> 1. I'd like to hear from the other core maintainers if they agree with
> this rule.
> 
> 2. I'd like to hear a rationale for this rule.

The number one rule is not to fork codebases in incompatible ways. Adding
new features to a particular version of Xen and not making any effort to
send them back upstream is needlessly forking the code. 

This ripples into libvirt, because if upstream then adds the same feature
but implements it in a different way, libvirt now has to satisfy two
conflicting implementations, which hurts long term maintainability of
our own code.

> 3. The Sun repository *is* the upstream for xend on Solaris for a number
>    of reasons:
> 
> a. The 3.1.x series is only maintained by us, there is no other 3.1 upstream
> 
> b. The XenSource upstream does not, and cannot, work on Solaris

Are you in essence saying that long term you don't intend to follow the
new Xen releases, and Solaris XenD should be considered a divergant
fork long term ? Hopefully this isn't the case, but it'd mean we have
to cope with the one libvirt Xen driver trying to serve & satisfy two
different masters at once.

> c. XenSource have no interest in xend at all and it's effectively
>    unmaintained (with the exception of Novell I think).

While they may not have been doing much active new development work,
they have still been accepting patches posted on xen-devel from a
variety of sources, and doing new maintainence releases of the code. 

> 4. I've seen stated a number of times that libvirt is designed to avoid
> becoming a lowest common denominator interface. I cannot reconcile that
> claim with this rule: all xend implementations must apparently be the
> same, and no enhancements are allowed.

This is not about lowest common denominator. It is about not needlessly
forking code bases when adding new capabilities. The network device 
unplug enhancement is a perfect example. This is not Solaris specific
in any way. If this were sent upstream and merged it serves to benefit
all users of Xen, and means libvirt only has one implementation to
worry about. If it is not sent upstream, and we put the Solaris speciifc
code in place, and upstream then implements it in a different way, libvirt 
now has to carry 2 different impls of network unplug code. This is really
not nice.

> 5. This rule further restricts innovation in that XenSource must agree
> to any changes. This has a rather obvious chilling effect. The distro
> type/variant recording changes are a perfect example: this is essential
> information, and recording it in xend was pre-NAKed by XenSource a long
> time ago.

IMHO, that's life in open source development. Sometimes things get nack'd 
and have to be implemented in a different manner to be acceptable to all
members of the upstream community. Having everyone go off and do their
own implementation of features when upstream reject them, is just not 
long term sustainable to the downstream users of that project being
forked.

> 6. As a thought experiment: if someone re-wrote xend in C, what then? Is
> that upstream, or not? In case it's not clear, all of Sun's Xen changes
> are open source and easily available.

It would likely be a totally different  product, requiring a from-scratch
libvirt driver to support it. FWIW, we already have this scenario in the 
form of XenAPI + XenEnterprise server, which ditched current Xen RPC api
that we use. If we added XenAPI support to libvirt, I expect it would be 
a completely new driver, whose code is independant of the existing libvirt
Xen driver code.  

This would actually be an easier situation, because there'd be an explicit
clean division between the implementations, and (within the confines of the
libvirt API semantics & XML format) can evolve independantly of each other,
not having to worry about one causing trouble for the other & vica-verca.
The problem we have currently is harder two reconcile, because it involves 
trying to make 1 driver work with 2 potentially divergent projects. 

> 7. Any change that involves *incompatibility* is clearly different, and
> IMHO not acceptable. This is only about changes that in no way
> negatively affect behaviour with an upstream Linux xend.

You need to consider future incompatability as well as any current
incompatability. ie the scenario where you implement feature XYX
one way, and the next upstream release implements it a completely
different way. libvirt now has to carry 2 different implementations
of the same code indefinitely, or choose to not support one of the
forks. Neither is a good option really. 
 
> 8. I hope it's clear to everyone that I'm motivated to merge changes
> upstream wherever feasible. I'm not seeking some kind of get-out clause
> from being a responsible maintainer.
> 
> 9. Practically, we will have to carry these changes in our libvirt no
> matter what. I'd rather not.
> 
> 10. I would note that Red Hat own the upstream for the code they care
> about, thus this rule can only cause trouble for other people.

Red Hat / Fedora always follow the rule of getting patches accepted by
upstream Xen developers, despite fact that RHEL-5 is on ancient XenD
3.0.3, new features for 5.1, 5.2, 5.3, updates are all required to be
upstream in xen-unstable first, and then backported. Likewise when we 
need new features for QEMU/KVM, we have to work with those development 
communities to get them to accept the required changes in QEMU codebae.
On many occassions I've had code rejected outright, or rejected needing
a re-write. The same rules for including new code in the QEMU driver of
libvirt - if it needs a new QEMU command line argument, or monitor 
command, then that must be present in upstream QEMU/KVM codebase first.

KVM is actually rather a PITA in some cases, because it has forked QEMU
and added features, which then got re-implemented in QEMU. We got 
horribly burned by this in the case of migration whose command line
syntax was changed. We should never have added support for that in
libvirt while it was still in KVM only because the long term pain it
caused outweighs the short term gain it had.

NB, once a feature is upstream, then it *is* acceptable to backport to an
existing release, because at that point you know that the patch will be
short-lived until the next code rebase & is unlikely to be changed in an
incompatible way. And downstream users of that feature (such as libvirt)
only have one implementation they have to worry about.


Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|