Log from todays (20080130) EPEL SIG Meeting

[Michael Schwendt]

On Wed, 30 Jan 2008 21:23:48 +0100, Thorsten Leemhuis wrote:

> 00:09:26 <       knurd> | so, how to move on? ask mschwendt to install is? nirik, or do you want to handle that? you likely have the needed permissions as well
> 00:09:42            --> | mdomsch (Matt_Domsch)  has joined #fedora-meeting
> 00:10:11 <      smooge> | mdomsch, has permission to. he can do anything :)
> 00:10:17 <       nirik> | sure, I can ask him if we can just run it on the buildsys box, or what.
> 00:10:27 <       knurd> | nirik, k, thx

> 00:23:51 <       nirik> | knurd: I mailed mschewnt asking about setting his script up on the buildsys box

So, here are some public comments on that. Didn't know this was a topic
in the meeting once more.


For Extras we used to execute some nohup background jobs in the post-push
hook (near the bottom of the pushscript config file). That was the primary
way of running scripts, particularly when pushes were done almost daily.
[To push without re-running such stuff, there's option -s which also
skips the repoprune and repoview steps, btw.] In the signer's group env
one must be careful, though, and work around permission problems in some
Python generated files (e.g. urlgrabber files created by yum) if they
are shared.

Running fully automated hardly ever was a good idea because Core and
Extras were not pushed at once and could be out-of-sync. Whether to
include or exclude updates-testing was an unresolved matter, too. Later I
switched to preventing broken deps by excluding packages from needsign
with the help of the RCNeedsign.py script. That got rid of the public
summaries and only mailed packagers privately (warning about whether
needsign and/or test updates were included).

To run the code as a cron job somebody first needs to spend some time and
determine under which circumstances it would run into error conditions.
Such as metadata loading problems (also adding the lockfile check for the
Plague needsign repodata lock), and then try to catch all errors. Otherwise
there would be the risk that it spams packagers with wrong reports.
Mission objective: also check yum api's error return codes.

I haven't done that extra work because of the Core+Extras+Test Updates
split (see above) and because it has never been agreed on how often to
mail broken deps reports to owners and the list. The initial work-around
for repo I/O problems was to sleep 12 minutes when repoclosure returned an
error and retry a few times. That seemed to work for cron as well as the
background jobs. The old scripts are in fedora cvs, so copying existing
bits from them would be possible.

The old repoclosure code is integrated into the pushscript and the
RCNeedsign.py module -- [That's the one that installs the pkgs from the
needsign queue into temporary repos to allow for proper
ExcludeArch/ExclusiveArch handling prior to running repoclosure on
that. Originally I wanted to test adding multilib support to it but didn't
continue when koji+bodhi appeared on the radar.]

The additional Extras repoclosure scripts on the buildsys machine are a
bit specific to Extras. Not all constants are fetched from the Config_*.py
files. E.g. the script needed to distinguish between Core and Extras (it's
a similar thing for for CentOS-or-RHEL and EPEL) through a repoid keyword,
hardcoded in the rc-report.py script. Which package owners list to read
was hardcoded, too. A few other features were added without any way to
configure them (e.g. the history support that printed the age in days of a
broken dep or the function that only sends a summary to the list if there
are new reports). The used Yum is a private copy with the
checkForObsoletes patch applied.

The recently published repoclosure tarball is easier to modify. It doesn't
depend on the pushscript code or config files. It takes an ordinary
yum.conf file as input. It can be configured with Fedora Account System
account details for package owner db access outside fedora infrastructure.
All it needs is Yum post 2.6.1 with checkForObsolete.


Whether and how often to run a script like that, whether or not to include
"testing" or needsign (*without* multilib, ExcludeArch, Exclusive}Arch),
whether to run against CentOS or the private RHEL repos, I don't want to
decide. I would appreciate if somebody else took over. Sometimes people,
who have a fixed package in updates-testing already, complain about a
broken deps report that doesn't cover updates-testing. On the other hand,
some people complain if a report covers updates-testing. Furthermore,
there are packagers who believe every report to be false. The only fun of
doing repoclosure stuff is to see those people who understand the reports
and fix their packages silently as soon as they receive a report.