[Fedora-packaging] time for a review-oh-matic?

[Daniel P. Berrange]

On Fri, Oct 03, 2008 at 12:21:04PM -0700, Chris Weyl wrote:
> So, building on the spirited debate going on with respect to a
> suggested review template, is it time for a review-oh-matic type
> service?
> 
> Many of the things we do for reviews are the same.  Check it builds in
> mock (koji scratch), check the srpm sources against upstream, look @
> rpmlint, look at the file lists, etc, etc...  Wouldn't it make all our
> lives to have a simple little tool that a packager could submit a srpm
> for review to.  It could then, say (and off the top of my head):
> 
> 1) extract the spec, and push it + the srpm somewhere
> 2) file a review bug
> 2) kick off a koji scratch build
> 3) post a link to the build, success/failure, build logs (a la the
> FTBFS overlord)
> ==> success, pull the built rpms, and post to the review bug:
> * rpmlint, requires/provides
> * md5/sha1 of srpm provided source against upstream
> * a partial template, good bad, etc.

If you want to get really clever make a database to back this whole
thing. Automate as many of the review points as possible and put the
results into the database. Leave humans to review the points which
need intelligent thought, and have some way to record their decisions
for each point, if clarification was needed. eg for an unusual 
license the DB might storage a comment providing justification for 
the license being valid in Fedora. Or for each rpmlint check record
the pass/warn/error state, and for ones which don't pass again allow
for the review to addd a comment justifying why the warn/error is ok
to be ignored, or why it must be fixed.

Now fast forward a short while, and have it automatically re-run all
checks for the automatable review points and report on changes. We
focus alot on initial review, but have very little, if any, oversight'
for ongoing package changes. Plenty of specfiles bit-rot and drift out
of compliance with review guidelines.

Of course this is all a huge job and I'm not volunteering to write
it, but I think more formal automation in the package review process
would be very beneficial, particularly if it could help us track or
identify changes post-review.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|