Fedora 12 QA retrospective - feedback needed

[Adam Williamson]

On Thu, 2009-11-26 at 12:28 -0500, James Laska wrote:

> Highlights for me ...
> 
>       * The issue was discovered prior to release ... that's 'a good
>         thing' [tm].  Just as cool, it was also discovered by someone
>         outside the core QA team
>       * The problem was correctly identified when filed by Alexander,
>         but the impact to the default F-11 preupgrade user wasn't known
>         at the time
>       * Preupgrade is a great application, was an opportunity to
>         identify failure scenarios missed when we (the royal 'we' ==
>         Fedora) chose it as a official upgrade method?
> 
> Did I miss any?

yeah, we did catch that one. I think the only really obvious scenario we
missed is one we already adjusted the test cases for - updating from a
realistic previous-release configuration, not a brand new clean install.
The problem is that there's as many potential failure cases as there are
combinations of packages and (especially) third-party repositories and
software, and there's a lot of those. Also different disk layouts and
bootloader configurations. I don't know how many we can realistically
expect to test, or where we'd want to draw the boundaries. There have
been quite a lot of people on the forums running into issues with
preupgrade for various reasons.

-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net