Best way to sign packages before adding to the repos?

[Dan Williams]

On Fri, 2005-11-18 at 10:08 -0500, Chris Weyl wrote:
> On 11/16/05, Dan Williams <dcbw at redhat.com> wrote:
> > The way Fedora Extras does it right now, there's a cron job or something
> > that pushes the built RPMs to the real repository directory and signs
> > them at that point.
> >
> > If you want to use the repository the build server dumps built RPMs
> > into, then you can use the repo scripts feature of the build server.
> > There's a config option in each target file on the build server which
> > accepts a path to the repo script.  That script is run after RPMs have
> > been copied to the repo, and is given with 1 argument": the target
> > string for the repo, which takes the form of something like
> > "fedora-development-core" or "fedora-extras-4".  You could probably do
> > the signing from that script.
> 
> Ok, that makes sense; it seems to catch it right before adding it to the repos.

Hmm, it actually runs the scripts _after_ packages have been copied to
the repository.  So maybe it's not exactly what you want.

> > Some issues to know about repo scripts:  they are called every time
> > packages are added to the repository.  That means, if you want to do
> > stuff only at certain times, like midnight, you have to do time tracking
> > yourself to make sure your script is only called every 24 hours.  Also,
> > the build server blocks while the script runs, so it will kill the
> > script after 1 hour of runtime to make sure that stuff doesn't block for
> > too long.
> >
> > This part hasn't been too tested or fleshed out, so if you think of
> > ideas for improvements, feel free to propose them.
> 
> Hmmm...  One thing that comes to mind, is that it would be useful if
> as a second & third parameter, the repo_script took the name of the
> package, and version-revision-etc that had just been built.  That
> could save time, either by avoiding any magic to figure out which one
> had just been completed, or by allowing brute-force approaches caused
> by not wanting to create said magic:)

Well, you may be adding 20 packages to the repo, or just 1 at the point
the script gets called.  But this would be helpful, yes.  Ideally I
guess we'd give the target string as the first arg, then paths to all
the RPMs that just got copied to the repo as the next 50000 arguments.

> Another thing, if the packages have already been added to the local
> repos when the build state is "needsign", then what's the point of
> that step?  If we go and sign them, then we need to play with the
> repos cache/etc to maintain a valid state; if we pull them out and
> push them to another (signed/etc) repos, then it's not needed; if we
> don't sign them it's not needed.  IMO, it would be far more useful if
> packages held in this state weren't actually added to the repos until
> after manually "finished".

The problem here is that if packages are held until manually finished,
then they are not available to builders.  So if I want to build package
B that depends on a new package A, I have to wait for you to sign
package A and add it to the repo before I can build package B.  That's
the main reason to add packages to the repo ASAP, and also to run the
repo scripts _after_ copying packages to the repodir and running
createrepo.  It's also probably why Fedora Extras doesn't use the build
server's repository but requires manual runs of a push-script to sign &
copy to the "official" mirrored repo.

It's a balance of package maintainer's needs (quick building of deps)
versus administrator needs (making sure repos are up to date and
signed).  Suggestions on how to better balance these two issues would be
welcomed.

In the longer term (plague 0.5) I'd like to put automatic depsolving
into the build server, so that the server will wait like 8 hours for
dependencies of a particular SRPM to be solved before failing the
package.  If during those 8 or 12 hours, the deps get solved, then the
build server will allow the package to be built.  This way, we let
package maintainers queue up however many packages they want, and then
walk away and not have to worry about stuff until they get mails about
failures or successes.  That's the end goal here.  And it should solve
your problem too, since it fixes the dependency issue that forces repo
scripts to be run last.

Dan