Fedora.us QA (was: Re: Prelink success story :))

Michael Schwendt ms-nospam-0306 at arcor.de
Fri Feb 27 12:43:51 UTC 2004


On Thu, 26 Feb 2004 22:03:26 -0500, Erik LaBianca wrote:

> > Reviewers should develop a feeling for
> > what is important and what is not.
> > 
> 
> How is a new reviewer supposed to develop that feeling? You've got to
> have a place to start.

You do. What's in the Wiki and what you run into upon reviewing packages.
For instance, you take a look at a package which contains a spec file with
multiple scriplets. That's something not covered by the current checklist,
and it would raise your interest, wouldn't it? In particular, because
programs executed in the scriplets may need explicit dependencies.  Or if
a package fails to build, wouldn't you want to find out why it fails?
Because if you didn't, the packager might reply "works for me", and you'd
be in a loop if it is non-obvious breakage. Or you see lots of explicit
dependencies, "Obsoletes:" or "Conflicts:" (not covered by the checklist
either).

> No they don't. However, as a newbie, my impression was that I was to
> review ALL aspects on the checklist. After doing some QA and looking at
> other peoples bugzilla reports I figured out that it would be ok if I
> just downloaded the package and made some comments about what I didn't
> like.

Sure, as long as you feel good about it. Notice that the packager
doesn't need to agree with what you don't like. ;o)
 
> However, people just making comments about what they like doesn't
> provide a deterministic path to REVIEWED status.

No, that's why there are the publish criteria. A section in the Wiki
mentions what a clearsigned review must contain.

> If I just make comments about broken things, and then submit an
> approval, how do you know I verified md5sums?

Never. How would I know whether you visited upstream actually and
not just ran md5sum on the included tarball?
 
> If the package works, let it go or
> argue about it, but don't stall it in QA over this.

That's why I point out that reviewers' comments usually are a mixture of
must-fix and should-change items.

> I'm seriously concerned that packages be able to move through fedora
> quickly and efficiently, without compromising their quality. I'm not
> convinced the current process is getting that job done.

It's not due to the process but due to lack of reviewers.
 
> > > 11. Does it pass rpmlint cleanly?
> > 
> > Not always possible. :)
> > 
> 
> Ok true, but if it doesn't maybe that's time for a more knowledge person
> to approve that which doesn't pass rpmlint?

Depends on what rpmlint reports. We package for Fedora Core, not to
make rpmlint happy.
 
> but
> expecting packages to be perfect before they pass QA is going to stall
> the process indefinitely. If those issues come up, the newbie may not be
> able to detect them. They'll get noticed sometime, and should get fixed
> then.

But we face an all-or-nothing mentality. People don't review because
they want to avoid mistakes.

> > NEEDSWORK is to move packages out of the QA queue, because they don't
> > build or need more work before they are moved back into the QA queue.
> > This can also be set by the packager if major changes are planned.
> > REVIEWED is the new keyword to flag reviewed packages, so they are
> > easy to locate.
> > 
> 
> OK. So at what point do I make the determination that the package has
> passed QA?

When you feel good about it. ;)

> This way my likelihood of looking like an idiot is very low, if I
> followed the instructions properly. Not feeling like an idiot is what
> makes people come back and feel good about contributing.

There is no silver bullet. I don't understand why/when you would feel like
an idiot.

> > issues. Sometimes you need to wait till the next weekend. If the packager
> > doesn't respond, simply move the request out of the package queue by
> > deleting the QA keyword and setting the NEEDSWORK keyword.
> 
> 
> OK, that's fine, but in all reality I'm uninterested in removing the
> package from the QA queue. I want the package published! And now! At
> what point is it ok for me to just post my own version of the SRPM? 

That cannot be covered with a rule. Talk to the packager, i.e. the current
maintainer or the one who submitted the package. You build a team with him
as soon as you comment on the package. If he doesn't mind someone else
taking over the package, you switch roles, and then you would be the
packager who needs reviews. :) But if either party has no time to work
on a package/review, patience is needed. Or additional reviewers who
help.

> Also, documentation of these keywords was not apparent to me in the web
> of pages I looked at while trying to figure this process out.

Some of the keywords don't really belong to a "process", but just help
upon classifying and locating bugzilla tickets.

> > If there are people with interest in a package, surely there should be
> > two reviewers, too?
> > 
> 
> I'm sure they are out there, but we've got to make the process so darn
> straightforward they can't help but succeed once they decide to help
> out. I'm skeptical that it's that way now. I'm not sure if I succeeded
> yet, and I can assure you I attacked it unsuccessfully a few times
> before finally making it over the hump of actually posting a review
> comment.

We need more reviewers and more feedback.

-- 





More information about the fedora-devel-list mailing list