[Freeipa-devel] [PATCH] setup.py for freeipa-python

[William Jon McCann]

Hi Karl,

On 10/11/07, Karl MacMillan <kmacmill at redhat.com> wrote:
> On Thu, 2007-10-11 at 11:08 -0400, William Jon McCann wrote:
> > Hi Rob,
> >
> > On 10/11/07, Rob Crittenden <rcritten at redhat.com> wrote:
> > > William Jon McCann wrote:
> > > > Hi,
> > > >
> > > > Here is a patch that adds a setup.py for the freeipa-python package.
> > > >
> > > > This makes it easier to build tarballs, rpms, and do installs.  It can
> > > > even generate a spec file:
> > > > python setup.py bdist_rpm --spec
> > > >
> > > > Jon
> > > >
> > >
> > > I'm not familiar with the python distutils package, what does this buy
> > > us? We already generate tar.gz and rpm files.
> >
> > Well, in my opinion, freeipa/Makefile really isn't a good build
> > system.  It has a number of problems:
> >
> >  * It exists outside of the discrete packages
> >    If someone wants to build or package one of the individual modules
> > either from hg or from a tarball it isn't straighforward
>
> Really? What doesn't work? The top-level makefile is supposed to just be
> a convenience for developers for building everything and making
> packages. And the individual tarballs get built whenever the src rpms
> are build.

Which is certainly not typical.  It doesn't work for at least the
following situations:
 * I want to make a tarball for ipa-python
 * I don't have RPM
 * I don't want RPM only SRPM
 * Want to compile .py files
 * etc, etc

> >  * The version numbers of the individual packages isn't contained in the package
>
> They are in the spec files - where else should they be?

In the build system or in the code.  Specifically not just in the
distribution metadata.

In the case for python modules, in the setup.py or in a config file.
In the case for autotools, in the configure.ac.

> >  * It is too heavily biased toward RPM production and not extensible
>
> Well - the lower-level makefiles are just plain makefiles. Again, the
> top-level is more just a convenience for developers - i.e. me :).

Right.  I think we can make both better.  We can make each standalone
system better and have a smarter way to pull them all together - for
those who need to build them all at once (probably only you :) )

> >  * The dist target builds RPMs and SRPMs
>
> what should it build?

It depends, of course, on the platform.  In the typical autotools
setup it will build tar.gz or tar.bz2.  With distutils you can build
these too or RPMS, SRPMS, and is extensible...

http://docs.python.org/dist/source-dist.html
http://docs.python.org/dist/built-dist.html
http://docs.python.org/dist/node33.html


> >  * Does not seem to detect python versions or directories
>
> It should - e.g.:
>
> PYTHONLIBDIR ?= $(shell  python -c "from distutils.sysconfig import *;
> print get_python_lib()")

Not in the toplevel it doesn't.  And even for ipa-python setup.py does
it way better.  Take a look.

> >  * Does not have a mechanism for specifying files to include/exclude
> > from distribution
> >
>
> What would that be used for?

That is used all the time in other build systems.

 * Exclude SCM metadata
 * Exclude built sources
 * Exclude temporary files
 * Exclude generated files
 * Exclude backup files
 * Allow for a "distcheck" target to be possible
 * etc

> > The Makefiles in the individual sudirectories have their own problems.
> >  It is probably sufficient for making the official tarballs and rpms
> > for the project but it is very inconvenient for an external
> > developer/packager to use.
> >
> > I would like to improve the overall system.  Since the subdirectories
> > are packaged and installed separately I think it makes sense for each
> > to be self-contained.  This makes it way easier to work with, hack on,
> > and test each package individually.
> >
>
> I'm having trouble understanding exactly what problems you are
> encountering. I'm definitely open to improving the build system - it is
> currently just the easiest possible thing we could do, so it has many
> shortcomings. However, understanding what you are looking for would help
> choose the right direction for the changes.

Sure, no problem.  I should be able to pull ipa-python from hg, hack
it, install it to a directory of my choice, build a tar.bz2, generate
a .spec.

> > Looking at ipa-python specifically:
> >
> > Since this is a pure python package, I think it makes sense to use:
> > http://docs.python.org/lib/module-distutils.html
> >
> > For one, it is the way people expect to work with a python library.
> > The Makefile that is used is a little lacking:
> >  * doesn't compile the .py files
> >  * maintain a version number
> >  * generate the .spec file
> >  * have a dist (tarball) target
> >
>
> I'm not a huge distutils fan. At the very least, the spec files it
> produces are not usable as a basis for packages to be included in Fedora
> (and likely other distributions), so we would end up maintaining those
> by hand anyway.

Oh.  Well very few packages actually maintain the distribution
specific metadata in their SCM.  The fact that you are is really an
artifact of being a Red Hat sponsored project.  Fortunately, I'm a
RHEL customer but I think that there is a real advantage to IPA
becoming a true "upstream" project.

> > I suppose another option would be to use autotools but I don't think
> > that makes sense for a pure python module.
> >
>
> I'm open to autotooling everything - but I have no experience myself.

For the stuff with C code it is probably more natural than distutils
anyway.  If this is the direction you'd like to go I'll give it a
shot.

Jon