QA process was Re: RPM submission procedure

Panu Matilainen pmatilai at welho.com
Fri Jan 9 18:05:06 UTC 2004


On Fri, 9 Jan 2004, Michael Schwendt wrote:

> On Fri, 9 Jan 2004 18:13:13 +0200 (EET), Panu Matilainen wrote:
> 
> > On Fri, 9 Jan 2004, Michael Schwendt wrote:
> > > On Fri, 9 Jan 2004 11:54:51 +0200 (EET), Panu Matilainen wrote:
> > > 
> > > > The fact that you need two such 
> > > > users outside of "trusted developers" to get the package *anywhere* 
> > > > doesn't do much to encourage people to do QA because it feels like totally 
> > > > wasted effort since the package isn't still moving. 
> > > 
> > > But if the first reviewer has done most of the work already, a second
> > > review can be like a piece of cake. Or is everyone waiting for someone
> > > else to be the early bird?
> > 
> > Sure it's a piece of cake, but *somebody* has to do it. If you look at the 
> > average package in fedora.us QA queue there are *very* esoteric packages 
> > in there.. in many cases you'll be lucky to find just that one person who 
> > understands/is interested enough to learn what the package is about *and* 
> > willing to do the few steps of QA review.
> 
> And there are no two users who can build and test-drive such packages?
> 
> I mean the packager does the low-level stuff. And the more "esoteric" a
> package is, the more familiar he ought to be with it, shouldn't he?  It
> would just need an "okay" from two users that the built software works and
> that the included sources have been verified with any available
> signatures, trusted sources, via diffs, with confirmation from upstream
> authors or other ways. We should not assume that every packager's work
> needs to be verified at a low level. Actually some packagers get it right
> from the early beginning. Other even stick very close to the spec
> template. Whether the software works and whether upgrades don't result in
> regression, is much more important. Such QA can be and should be performed
> by actual users of a given piece of software. Let packagers and users
> team up. 

By all means, but the current fedora.us policies and practises hardly 
*encourage* that. The amount of nitpicking trusted developers produce 
(among themselves) is enough to scare off anybody starting in packaging 
I'm willing to bet :)

> There are no users? Ugh. Then who cares? Let the package
> rest in peace until someone steps up who's interested in it.

Sure.

> 
> > > With regard the notion of "untrusted/trusted developers", I favour the
> > > concept of giving "untrusted developers" the chance to take responsibility
> > > early by relying on their review. That means, if there's a package request
> > > ticket with a PUBLISH vote, I assume the stuff has been reviewed actually,
> > > so let's apply some of the mandatory sanity checks only, skim over the
> > > rest and get it published, taking the first "untrusted" review into
> > > account.  Heck, if that first review was half-hearted, there's still the
> > > PENDING step, and the packager shouldn't "sleep" either and be somewhat
> > > familiar with a package, too, before it is approved.
> > 
> > I agree - but the way it's expressed in our current policies doesn't come 
> > out sounding like that. At least to me, the policy sounds basically like 
> > "you have to do the same checks the other guy did" meaning pretty 
> > redundant and boring work somebody already did... So if others agree, we 
> > should at least rephrase the "publish vote from two untrusted developers" 
> > to something which expresses what you say above better than the current 
> > policy.
> 
> But we don't have a complete and accurate list of what must be included in
> a package approval, other than MD5 checksums of source tarballs and
> src.rpm, e.g. as mentioned here
> http://www.fedora.us/wiki/PackageSubmissionQAPolicy 
> 
> So how do you know what exactly a reviewer has reviewed? He could post
> checksums, vote for publish and be done. Sure, the QA checklist is
> available. But it's incomplete and includes steps not needed for many
> packages. Turns out it really only needs two people to approve a
> package. If no two people have interest in a package, can we expect that
> there would be any users of a published binary package? Would they submit
> bug reports or move on to another source of prebuilt package in case of
> problems?

Probably not.. or then they would. Do the freshrpms, atrpms, dag, newrpms 
"gang" get reports of faulty packages, without the users actually looking 
inside specs? Sure they do.

> 
> > > *Sometimes* the packager does most of the QA already, anyway.
> > > 
> > > > I've been lobbying lowering the entry bar to testing/unstable for a while
> > > > now... Stable repository should IMHO remain basically as it is, eg you
> > > > can't get your package there until it's seen a full QA review under
> > > > current rules. However allowing just one "untrusted" developer QA review
> > > > to get package into testing or unstable would at least give people the
> > > > feeling that their effort is valuable and actually helps. 
> > > 
> > > Bears the problem that people would be entirely satisfied with getting
> > > their packages published in testing/unstable easily and don't care to get
> > > them into "stable". It shouldn't end like RHContrib, but it would be
> > > worth a try: +1
> > 
> > Oh, I certainly don't want another RH contrib. I want to *lower* the bar
> > to get packages out in the field, not remove it :) People are not going to
> > be satisfied by just getting their packages into testing/unstable but it's
> > a *start* - it's much easier to get users to test something they don't
> > have to build themselves.
> 
> Hence my old suggestion that packagers offer not just src.rpms but also
> prebuilt rpms *if* users show interest in helping to approve them. Whether
> you trust such packages or not, is up to you.

That's what I do personally for the more popular packages like apt. The
problem with that is that it requires some place to put the packages into;
not everybody is willing or can afford to pay for the space and bandwidth
for the purpose. You could just as well have a common repository (be it 
RPMS.unverified-use-at-your-own-risk or whatever) where each package is 
signed by the packagers GPG-key - if you choose to trust some packager
you import his/her GPG key and don't worry about getting whatever cruft 
out of that repo on your system. 

	- Panu -





More information about the fedora-devel-list mailing list