Fedora release criteria completely revised

Tue Dec 8 15:26:28 UTC 2009

On Tue, 2009-12-08 at 08:16 +0000, "Jóhann B. Guðmundsson" wrote:
> On 12/07/2009 10:55 PM, Adam Williamson wrote:
> > During FUDCon, we've been working on revising the Fedora release criteria.
> > John Poelstra had already fleshed out a structure and much of the final
> > content, and we've been revising and tweaking it in conjunction with QA
> > (myself, Will Woods and James Laska), release engineering (Jesse Keating),
> > anaconda team (especially Denise Dumas and Peter Jones) and desktop team
> > (Christopher Aillon and Matthias Clasen, who provided suggestions at an
> > earlier stage).
> >
> Is this discussion available somewhere online?

It was a hackfest, not a presentation, so it wasn't recorded - we were
moving around and talking to different people and things, it took a day
and a half, so it wouldn't have been practical. Remember it was
discussed for quite a while before FUDCon too.

> > The new structure is based around a general page and specific pages for the
> > Fedora 13 Alpha, Beta and Final releases (which have been written
> > generically so they can easily be converted into pages for F14 and all
> > future releases just by copying and pasting). You can find the criteria
> > here:
> >
> > https://fedoraproject.org/wiki/Fedora_Release_Criteria
> > https://fedoraproject.org/wiki/Fedora_13_Alpha_Release_Criteria
> > https://fedoraproject.org/wiki/Fedora_13_Beta_Release_Criteria
> >   
> 
> Category 10 in the Beta Release Criteria.
> 
> "The installer must be able to successfully complete an upgrade
> installation from a clean, fully updated default installation of the
> previous stable Fedora release, either via preupgrade or by booting to
> the installer manually"
> 
> I think we first need to get established if Fedora officially "supports"
> upgrades now.  If it does we need create and add several upgrade test
> cases and to do so we need to know what "default installation" is. What
> packages that installation contains and since Releng needs to provide us
> with that list it would be good if they document and explain at the same
> time how and why they choose to make that selection the default. (
> Default dvd install from my perspective should just be secure base no
> auto selection for the end user in the installer. )

'Supports' is kind of a vague term and not really useful. We don't
'support' anything in the sense of guaranteeing that it'll work - it's
not like we give you a refund if your system breaks :) So I try to avoid
the word when talking formally about this sort of thing.

What it basically means is that at beta stage, we test that if you just
install the previous release into a VM with all default options, update
it, and then run an upgrade, it works. This isn't the same as
'supporting' upgrades, because it only tests a single very simple case.
It's more of a basic check that the fundamental bits of upgrade code do
their job, and there's no really huge bug that everyone who tries to
upgrade is going to hit. We're not committing to testing and supporting
every conceivable combination of packages for upgrading.

This isn't really a change - we've always been doing this test anyway,
it's part of the installation test matrix, and if it had failed, we
would have considered that a blocker bug. A lot of what this page does
is just properly document and justify the tests that we've already been
running in QA, with no official policy document to define what tests we
_should_ run.

> Where there representatives from all the *DE GNOME KDE, XFCE, LXDE SUGAR
> MOBLIN ( I would not be surprise if I'm forgetting some ) present and
> chimed in on this? What where these suggestion that Christopher and
> Matthias and others made encase any one wants to chime in and provide
> additional feedback?

The initial session was mostly poelcat, me and jlaska working the tests
we have into the policy and cleaning up the high-level explanation
stuff. At that point it had all the anaconda stuff in it, plus a few
desktop things Matthias had suggested before FUDCon. We then took it to
the anaconda team for their feedback, and to Christopher Aillon for a
desktop team perspective, and added their suggestions. There were KDE
people at FUDCon and the criteria hackfest was announced so they should
have known about it, but they didn't come to offer suggestions and I
didn't run into any of them while I was working on it to ask for their
suggestions. Adam Miller was around and knows about this stuff from an
XFCE position, but again he hasn't given anything the XFCE team would
like to list. Moblin and Sugar are different as they don't get released
as part of the Fedora release, so they probably don't go into these
criteria.

You can see the suggestions made by Chris and Matthias pretty easily:
all the desktop-related criteria in the pages come from them. All the
stuff about what has to work in the 'default desktop' (so currently
GNOME) is feedback from them.

> > Desktop team - can you please let us know of any additional things that you
> > would expect to be working at each point during the release cycle? Note
> > that only things that *must* be working at each point should be listed on
> > these pages, not nice-to-haves. You must be able to commit to the idea
> > that, if any criterion on the page is not met, we would slip the release in
> > question.
> >   
> Was this sent to all the *DE lists? ( Noticed that only Gnome was cc on
> this ).

Unfortunately not yet, just because I'm not subscribed to them and I was
sending that message from sucktastic FUDCon wireless so it would have
been pain to subscribe to them to send it :) I will definitely get in
touch with each spin SIG to see what they want to feed back into the
criteria, though. The pages as they currently exist aren't set in stone,
though there is an issue about the relative importance of the spins
which we really can't answer in a discussion simply about the release
criteria, because it's a much bigger issue than that.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net