Updated Schedule for F12

[James Laska]

On Fri, 2009-07-10 at 17:12 +0000, Jesse Keating wrote:
> On Wed, 2009-07-08 at 17:00 -0700, John Poelstra wrote:
> > o Added earlier blocker review days (f13)
> > o Changed usage of "Release Candidate"  (f13)
> > o Fixed dates for Beta phase so that "to" and "from" dates are
> > correct 
> > in the context of the report (notting)
> > 
> > http://poelstra.fedorapeople.org/schedules/f-12/f-12-releng-tasks.html
> > 
> > Hopefully this version is a good starting point for the meeting f13
> > said 
> > he would arrange with QA at Monday's meeting.
> > 
> 
> Hrm, something is still missing.  Here is the feedback that I gave
> James.
> 
> 
> 
> Blocker Reviews) we need a blocker review day to happen with enough time
> for maintainers to react and fix things prior to the freeze.  I see
> there are ones scheduled for the 17th, 24th, and 31st, with the freeze
> being Aug 4th.  This is good, but is there a good reason to have 3
> before the freeze, rather than 2?  Just curious.  Also curious (and
> forgetful) what the reason is we are doing these on Fridays.  What's
> missing here is a review of the blocker bugs just before the RC compose
> (is this implicit?) as well as one /after/ the RC compose and testing.

All good points.  I'm not too worried about the number.  I think our
intent is in the right place to address the issues we had during F11;
more blocker bug days, and earlier.  

I don't have objections to adjusting the number or day of week.  Just as
long as we're getting more events before freezes + compose.  If we've
scheduled too many, we can easily scale that back next time.  Or it'll
provide a nice moment where we can breath amid the release storm :) 

> Test Composes) Releng would like to provide QA with at least one test
> compose with enough time for QA to test and provide feedback prior to
> the freeze.  This will help us and maintainers know what is broken and
> needs fixing, saving nasty surprises and slips once we actually do
> freeze.  On the current schedule I do not see any scheduled compose
> prior to the freeze.  This means that the first time we'd have an
> opportunity to test full media installs and reduced package sets, etc..
> is after we've frozen and when we're under extreme time pressure to get
> the release out.  High potential for slippage when operating this way.

Hmm. Not a bad idea.  There's risk both avenues here.  We can test media
prior to freeze and still discover media-related issues after freeze
since there's a large list of components where change can impact the
media kit.

But I think it's something worth trying for the next release.  How about
1 week prior to the freeze we schedule a physical media verification
run?  This would be a light touch test run focusing on boot and install
from physical media created from releng infrastructure (instead of
private test day created images).  As a start, the list would include:
      * Live image for the default desktop environment
      * boot.iso
      * DVD.iso
      * CD.iso

I can ask Liam if that's something to work into the existing install
test plan.  Would this meet the needs?

> Release Candidate)  Releng would like there to be only one scheduled
> release candidate.  It is difficult to schedule because we cannot create
> an RC until the blocker list is clear, see above needing a blocker list
> review just prior to RC compose date.  Looking at Alpha, RC compose date
> should probably be the 6th or 7th, to give QA enough time to test it and
> to have a chance at second RC if necessary on the 10th or 11th.  To have
> RC compose on the 6th, we'll need blocker review either on the 6th
> morning, or 5th evening.  Either we're clear and can compose RC, or
> we're not clear and we have to delay RC.  We'll need another blocker
> review on the 10th evening or 11th morning.  Either we're clear and we
> publish the previous RC, or there were new things picked up that need to
> be fixed and we get another RC, or there are too many things to fix and
> we slip.

I agree, the ideal scenario for QA is also the single RC.  The part that
really resonates for me is:

        Either we're clear and can compose RC, or
        we're not clear and we have to delay RC.

This puts the focus on planning and hosting successful blocker bug
reviews.  Which demands improved criteria around what bugs are valid,
which demands quality bug reports and effective screening/triage of
incoming reports.  QA has made good progress on these fronts and I think
putting more emphasis on the reviews will help flesh out additional
areas for improvement.

> I feel strongly that we shouldn't actually schedule a second RC.  That
> just gives too much to the perception that things can wait until the
> second RC to be fixed.  A second RC should be a disaster to be avoided,
> not a constant to be assumed.  However I do think we should setup the
> schedule to leave room for a second RC without a slip, should we need
> it.

I don't feel as strongly as the multiple RC's, but I agree 100% in
planning for those "oh shucks" moments.

I see these dates more as scheduled checkpoints.  These are points that
if we fail to hit them, they are good indicators we will miss a
following date ... and should consider possible corrective action.  A
successful release would be one where we deliver ahead of schedule ...
and begin staging with just the first RC checkpoint.

However, I do understand that having multiple scheduled release
candidates seems odd.  It does feel like an unusual use of the term.
This is why I like your suggested rephrasing to TC (test compose) and 1
RC.  That still doesn't preclude having multiple RC's should situations
on the ground demand it.  But that's okay ... that's why they're
candidates for release [1].  All bugs are evaluated through the same
severity + criteria prism.

> James, does this accurately describe what we discussed earlier?

Indeed, thanks for following up ... you quoted better than I could
have :)

Thanks,
James

[1]
http://en.wikipedia.org/wiki/Software_release_life_cycle#Release_candidate

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20090710/d319c9be/attachment.sig>