Fedora 12 QA retrospective - feedback needed

[James Laska]

On Wed, 2009-11-25 at 11:44 -0800, Adam Williamson wrote:
> On Wed, 2009-11-25 at 14:29 -0500, James Laska wrote:
> > Greetings folks,
> > 
> > Whether you call it a post-mortem, retrospective or lessons learned ...
> > the end result is the same.  I'd like to collect thoughts on how
> > good/bad of a job the QA group did in planning and testing the Fedora 12
> > release.  In keeping with the release-wide retrospective from Fedora 11
> > [1], feel free to share any wishlist items as well.
> > 
> > I've started the discussion on the wiki at
> > https://fedoraproject.org/wiki/Fedora_12_QA_Retrospective.  
> > 
> > Adding your thoughts is easy ...
> >       * Edit the wiki directly (instructions provided for ~anonymous
> >         feedback)
> >       * Or, reply to this mail (I'll collect feedback and add to the
> >         wiki)
> > 
> > Over the next week, I plan to organize any feedback and discuss the
> > highlights during an upcoming QA team meeting.  The goal will be to
> > prioritize the pain points and use as a basis for defining objectives
> > for Fedora 13.
> > 
> > Thanks for your feedback!
> > James
> 
> 1) in the end, we focused heavily on just three component areas for
> testing: anaconda, kernel, and X.org. This is primarily a function of
> the fact that these are the most vital bits; it feels like we're still
> at the point of doing 'let's make sure it's not totally broken' testing
> than 'let's make sure it's really good' testing. We didn't do stuff like
> making sure the desktop was polished.
> 
> 2) there was clearly a lot of uncertainty about RAID issues; it's
> something we obviously don't as a team test well enough (and some of us
> personally don't understand enough :>). In the end there didn't turn out
> to be any horrible issues, but the confusion was evident, and we did
> miss the Intel BIOS RAID stuff-ups. For F13 we should have better RAID
> testing both in Test Days and in pre-release test cycles.
> 
> 3) We weren't completely on top of X.org bugs for this release. The ones
> that wound up getting promoted to release blocker level were kind of an
> arbitrary selection. I think we got nouveau mostly right as I had a
> reasonable grip on nouveau triage, but we just had too few people to
> triage server / intel / ati bugs during the cycle, so when we hit beta /
> RC stage, we didn't have the whole bug set well enough triaged to be
> able to be sure we picked the most important bugs as blockers. For F13
> we should stay on top of triage better so we can do blocker
> identification accurately. happily, matej is more active on X triage
> again now and we have some more assistance from Chris Campbell (thank
> you Chris!) further volunteers would be great. I will try to stay on top
> of nouveau, again.
> 
> 4) we have the big security thing to deal with. I did start a thread on
> -devel about that.

I gather this goes under the heading "things that could have been
better?"  When you say 'security thing', do you mean that we don't have
a policy or plan to ensure a basic level of security (as your
f-devel-list thread raises).  Or are there other security aspects from
F-12 QA to consider?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20091126/a487ae2e/attachment.sig>