[libvirt] RFC: Splitting python binding out into a separate repo & ading to PyPi

[Daniel P. Berrange]

As everyone knows, we have historically always shipped the python binding
as part of the libvirt primary tar.gz distribution. In some ways that has
simplified life for people, since we know they'll always have a libvirt
python that matches their libvirt C library.

At the same time though, this policy of ours is causing increasing amounts
of pain for a number of our downstream users.

In OpenStack, in particular, their development and test environments aim
to be able to not rely on any system installed python packages. They use
a virtualenv and pip to install all python deps from PyPi (equivalent of
Perl's CPAN in Python world). This approach works for everything except
the libvirt Python code which is not available standalone on PyPi. This
is causing so much pain that people have suggested taking the libvirt
python code we ship and just uploading it to PyPi themselves[1]. This
would obviously be a somewhat hostile action to take, but the way we
distribute libvirt python is forcing OpenStack to consider such things.

In RHEL world too, bundling of libvirt + its python binding is causing
pain with the fairly recent concept of "software collections"[2]. This
allows users to install multiple versions of languages like Python, Perl,
etc on the same box in parallel. To use libvirt python with thse alternate
python installs though, requires that they recompile the entire libvirt
distribution just to get the Python binding. This is obviously not an
approach that works for most people, particularly if they're looking to
populate their software collection using 'pip' rather than RPM.

Looking on google there are a number of other people asking for libvirt
python as a separate module, eg on Stackoverflow[3].


I don't think these issues are going to go away, in fact I think they
will likely become more pressing, until the point where some 3rd party
takes the step of providing libvirt python bindings themselves. I don't
think we want to let ourselves drift into the situation where we loose
control over releasing libvirt python bindings.

IMHO we should / must listen to our users here before it is too late.

We can still release libvirt python at the same time as normal libvirt
releases, and require that people update the bindings whenever adding
new APIs (if the generator doesn't cope with them). We should simply
distribute python as a separate tar.gz, as we do for all other languages,
and upload it to PyPi, as well as libvirt.org FTP when doing a release.

Obviously there will be some work to separate things out, but I don't
see that being insurmountable, since all other language bindings manage
to be separate, even when doing code generation. We'd also want to
change to use distutils, rather than autoconf, since that's what the
python world wants.

Regards,
Daniel

[1] http://lists.openstack.org/pipermail/openstack-dev/2013-August/013187.html
[2] https://access.redhat.com/site/documentation/en-US/Red_Hat_Developer_Toolset/1/html/Software_Collections_Guide/
[3] http://stackoverflow.com/questions/14924460/is-there-any-way-to-install-libvirt-using-pip
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|