Fedora Release Engineering Meeting Recap - 2009-05-04

[John Poelstra]

Jesse Keating said the following on 05/06/2009 05:04 PM Pacific Time:
> On Wed, 2009-05-06 at 16:52 -0700, John Poelstra wrote:
>> The overall Fedora 12 schedule with a GA date of 2009-11-03 is 21 days 
>> shorter than Fedora 11 and exactly the same length as Fedora 8 so it is 
>> not unusually short.
> 
> Well 21 days is a lot of days to take in, particularly when a lot of
> them seemed to come between the GA of 11 and the Alpha freeze of 12.

I still don't understand why that matters considering the amount of 
development and changes that happen up until GA no matter where we are 
in the release process.  Even now after final freeze lots of changes are 
being ACKd on releng-list and very few if any are getting NAKd.

>>> way I could do that was to drop the Alpha cycle.  While already a
>>> non-blocking freeze, it still drew too much attention away from ongoing
>>> development in order to deliver something that was weeks old and already
>>> irrelevant.  The alpha has had dubious quality to the development cycle,
>>> at least from the developer and tester POV.  About the only thing of
>>> quality it provided was a "known good starting point" for which to
>>> install and then update to rawhide, and even that hasn't been true for
>>> large swaths of folks in recent releases.
>> I'd be curious to see our torrent and download numbers for the F11 Alpha 
>> to understand how insignificant it was.  Where can we find them?
> 
> Torrent numbers are at http://spins.fedoraproject.org:6969/  but
> downloads alone don't show the whole picture.  By the time people got to
> the Alpha bits, the general feeling I have is that most of the bugs in
> alpha were already known, and the solution was to update to rawhide.
> Even if the bugs weren't fixed, the first suggestion was always update
> to rawhide, so rawhide is really the target we're after, alpha just
> appeared to be an easy way to get there.

If I'm reading the information correctly there, 10,000+ people got the 
Fedora 11 alpha.

I would think that giving 10,000 people an easy starting place for a 
release would be a good thing.

How will we measure--not assert or subjectively decide as we are doing 
here and have done in the past--whether a change we have made helps our 
releases or if time is better spent somewhere else?

It seems that we continue to experiment and tweak the schedule with 
different durations, freezes, etc. but we rarely collect any hard data 
to evaluate whether or not past tweaks were beneficial or not.

If we can't really measure our progress or lack thereof in tangible 
terms, how do we really know we should continue to tweak things?

>> Although it is simple to just "not do the Alpha" it will take more 
>> coordination across the other Fedora teams AND good messaging in the 
>> press and wider world that no alpha is coming for Fedora 12 along with 
>> the reasons why. Have we considered the cost of this trade-off to our 
>> brand and community?
> 
> Hard to judge the cost, however our "three tests and a release" which
> turned into "Alpha Beta Preview Release" was really designed before we
> had things such as Test Days and the easily available live images, or
> the ability to easily compose regular install images with slightly
> different package sets.  The prohibitive cost to doing images on demand
> has gone down considerably, and combined with test days provides a
> better harness for gathering feedback as we go than the date based
> quickly stagnant snapshots.

This sounds good, but how do we really know this is true?  What data can 
we point to? Maybe we are focusing on the wrong problem and should 
instead focus on the amount of churn and changes during development.

John