[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Extracting python dependencies automatically, take two



On Thu, 16 Aug 2007, Panu Matilainen wrote:
The big problem with using modulefinder is that it expects to be able to actually import the modules. That might work fairly ok for the occasional "rpmbuild -bb my.spec" use on developers host, but in minimal buildroots... to satisfy that, the manual Requires turn into BuildRequires, so much for automatic dependencies. In other words, the current pythondeps.py is totally useless in real world :)

It gets worse from here. Things are fairly straightforward as long as we're only talking about modules in standard locations (either things in python standard library or site-packages) but what about cases where the software uses a custom path to import private modules (eg yum importing stuff from /usr/share/yum-cli)? Since you can't verify if something is in site-packages (because you can't assume them being installed on minimal buildroot), there's no way to know if import A is on private or public module. It's possible to look through the filenames and see if there are matching *.py[co] files in non-standard locations and assume they're just private modules but there is some room for error in there.

Largely rewrote the thing to at least try to address some of these issues, updated version again at http://laiskiainen.org/rpm/scripts/pythondeps.py

Modulefinder is now gone and replaced with manual parsing of imports from the python scripts. Only unconditional module initialization level imports are looked at, so any imports within functions etc will go unnoticed. OTOH this way most false positives are avoided.

There are still various cases where it gets confused but should be a fair bit better and also way faster because it doesn't actually import half the world when processing stuff. One known issue is with submodules within a single file, such as os.path in os module, that should be resolvable.

The much nastier issue is software using "private" modules of other software by extending sys.path, that's pretty much unsolvable automatically.

	- Panu -


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]