[libvirt] RFC: Splitting python binding out into a separate repo & ading to PyPi

[Mark McLoughlin]

On Thu, 2013-08-29 at 12:24 +0100, Daniel P. Berrange wrote:

> In OpenStack, in particular, their development and test environments aim
> to be able to not rely on any system installed python packages. They use
> a virtualenv and pip to install all python deps from PyPi (equivalent of
> Perl's CPAN in Python world). This approach works for everything except
> the libvirt Python code which is not available standalone on PyPi. This
> is causing so much pain that people have suggested taking the libvirt
> python code we ship and just uploading it to PyPi themselves[1]. This
> would obviously be a somewhat hostile action to take, but the way we
> distribute libvirt python is forcing OpenStack to consider such things.

As always, Dan is spot on.

You see little side effects of this problem in a number of different
places. For example, Nova has to enable "sitepackages" in its virtualenv
used for unit testing because libvirt can't be installed in the
virtualenv:

  https://github.com/openstack/nova/blob/86c97ff/tox.ini#L5

Or similar, here in TripleO:

  https://github.com/openstack/tripleo-image-elements/blob/e14861e/elements/os-svc-install/bin/os-svc-install#L16

Or there's the hack for running Nova's unit tests without libvirt
installed:

  https://github.com/openstack/nova/blob/86c97ff0/nova/tests/virt/libvirt/test_libvirt.py#L70

I've seen this come up again and again on the project and generally the
reaction is "gah! libvirt!" and while a workaround is found, it leaves a
bitter taste.

qpid was, up until 6 months ago, the only other that suffered from this
problem but (AFAIR) someone involved in OpenStack did the work to make
it available on pypi:

  https://pypi.python.org/pypi/qpid-python/

As Dan mentions, it's really quite possible that somebody will dive in
and come up with a temporary hack to get it up on PyPI.

I know this will take some significant work to get right, but it will
make a huge difference for OpenStack and libvirt's perception within
OpenStack.

Thanks,
Mark.