plague: Job waited too long for repo to unlock. Killing it...

[Dan Williams]

On Sun, 2007-12-30 at 17:54 +0100, Michael Schwendt wrote:
> If in a failed job.log you see the message
> 
>     Job waited too long for repo to unlock. Killing it...
> 
> please notify me.
> 
> It's a problem in the plague server code that results in a denial of
> service for subsequent build jobs. I have a traceback from Dec 28th, but
> in the context of the source code it doesn't make sense yet (because a few
> lines earlier the code ensures that the files to be copied exist and are
> readable). Buildsys runs a slightly modified version that adds a bit more
> debug output in this area.

Maybe just trap the exception, print it out, and continue?  That way at
least the server doesn't fall over, it just fails to copy one item.  It
might also help debugging to see if only specific files can't be
copied...

Dan

> [...]
> 
> 37683 (grads/ppc): Build result files - [ 'state.log', 'grads-1.9b4-21.el5.src.rpm', 'grads-debuginfo-1.9b4-21.el5.ppc.rpm', 'grads-1.9b4-21.el5.ppc.rpm', 'root.log', 'build.log', 'job.log' ]
> 37683 (grads/x86_64): Build result files - [ 'root.log', 'build.log', 'grads-debuginfo-1.9b4-21.el5.x86_64.rpm', 'grads-1.9b4-21.el5.x86_64.rpm', 'grads-1.9b4-21.el5.src.rpm', 'state.log', 'job.log' ]
> 37683 (grads/i386): Build result files - [ 'build.log', 'state.log', 'grads-1.9b4-21.el5.i386.rpm', 'job.log', 'root.log', 'grads-debuginfo-1.9b4-21.el5.i386.rpm', 'grads-1.9b4-21.el5.src.rpm' ]
> Repo 'fedora-5-epel': updating repository metadata...
> Exception in thread Repo: fedora-5-epel:
> Traceback (most recent call last):
>   File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap
>     self.run()
>   File "/usr/share/plague/server/Repo.py", line 233, in run
>     self._update_repo()
>   File "/usr/share/plague/server/Repo.py", line 137, in _update_repo
>     shutil.copy(src, file_in_dst)
>   File "/usr/lib64/python2.3/shutil.py", line 72, in copy
>     copymode(src, dst)
>   File "/usr/lib64/python2.3/shutil.py", line 49, in copymode
>     st = os.stat(src)
> OSError: [Errno 2] No such file or directory: '/srv/rpmbuild/server_work/fedora-5-epel/37683-grads-1.9b4-21.el5/x86_64/grads-1.9b4-21.el5.x86_64.rpm'
> 37683 (grads): Job finished.
> 
> --
> Fedora-buildsys-list mailing list
> Fedora-buildsys-list at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-buildsys-list