Is the push window too big?

[Michael Schwendt]

On Thu, 08 Jun 2006 10:00:59 -0400, Dan Williams wrote:

> Any idea how long each push takes, for a reasonable # of packages?  

In total, a lot longer than 30 minutes. :(

> Dave
> Woodhouse just ran into a repo timeout, where the repo was locked longer
> than 30 minutes and the builder killed the job (intentionally and
> correctly) because the server hadn't told it to unlock yet.

That doesn't sound as if it's related. The push script does not lock
anything for such a long period. The new locks below repodir are locked
only as long as it takes to type in the passphrase, sign and move the
packages and clean up empty dirs. For an average push, I believe that
takes somewhere below one minute in total per dist.

Only after that the time-consuming operations are performed, and no repo
below repodir is locked at the time, as they are not needed anymore.
 
> If the potential window is > 30m, we should increase the timeout in the
> builders to account for that.  I just picked 30m as a default with no
> particular idea if it was correct or not.
> 
> Timeout should account for both the server's copy+createrepo run, and
> the Extras push script operations.

Reading the bugzilla ticket Thorsten has pointed to, all this smells
much like we should try the copy-and-mark-as-done approach.

In pseudo-code, for each dist X:

 1 - lock plague-results repo for dist X
 2 - examine available new builds which are not marked as PUSHED
 3 - copy available builds to local master repo
   - keep track of what packages we create in there
 4 - sign all new packages in local master repo
   - in case of error, rollback, i.e. revert to state before (3)
     by removing the packages we tracked
   - if successful, mark all copied builds as PUSHED
 5 - unlock plague-results repo for dist X

If this 5-steps transaction is successful, the usual repomanage,
createrepo, and repoview stuff is run. A cleanup job can remove old
packages, which are marked as PUSHED, from the plague-results repo after
one or two days.

With this approach, we remove builds from the plague-results repo only
when they are available in the master repository. And we don't sign/modify
rpms in the plague-results repo, so this doesn't confuse the build servers
either.