Testing test releases: do not update

Mike A. Harris mharris at redhat.com
Thu Feb 26 22:16:05 UTC 2004


On Tue, 24 Feb 2004, Robert P. J. Day wrote:

>> I think that Robert Day's initial point is correct. If there is no
>> stable baseline then testers are constantly finding superficial bugs;
>> deep bugs that take hours of testing will never get reached. Alan Cox is
>> also right when he states that a tester should check against the current
>> state of rawhide before he reports a bug.
>
>the more i read posted on this, the more i'm coming to sort of
>understand the other positions so, yes, i'm starting to
>understand the philosophy of, this early in the release process,
>just drive all of the changes out there and see what breaks.

That's how it has been for 10 years.  I'm not sure why some
people think the processes that have been used all this time are
all of a sudden bad.  Perhaps people just do not understand the 
procedures and what to expect.


>but i guess it means that, unlike previous test releases from
>red hat, these early test releases really are unusable as
>*anything* but test platforms.

Ah, but in the past, the public only got to see beta release 4, 
5, and 6.  The first 2-3 betas that destroyed the universe, were 
not public, and so you did not see the level of breakage that 
occured in earlier betas.  Now things are even more open, and 
people who elect to see the level of breakage in early betas, can 
now do so.  Those who do not want to see high level of breakage, 
should really sit out a few test releases and wait for the OS to 
stabilize.  Or should only test low-risk packages individually, 
and not make changes to their core OS that could break 
everything.

>as i mentioned in a previous post, it used to be that ambitious
>testers would just flat out install the test release, knowing
>that it would have problems, but they'd be prepared to deal with
>that, *knowing* that as bugs were reported and patches issued,
>their systems would slowly get more and more stable, and their
>lives would return to normal.  well, as normal as testers' lives
>get.

In the past though, as indicated above, the initial betas weren't 
public, and so you never got to see just how broken things can 
get.  ;o)


>but with this new fedora approach, that's just not true anymore,
>at least for the first release or two.  if one is constantly
>updating against rawhide, then you have to assume that, as some
>things get fixed, others will get broken.  which makes it pretty
>much impossible to use such a system for useful work, no?  not a
>complaint, just an observation.  :-)

That's a fair observation, but you should be aware that this is
absolutely no different than any previous OS release, other than
the fact it is an open process now.  We _NEED_ wider testing than
we can do internally alone in order to get things in a more
stable state.  That either means we do private beta releases with
a selected team of individuals who 100% accept the deal about the
chances of having a totally broken system for the private betas, 
or we make it open, and let people decide for themselves.  We 
chose the latter.  One thing we can _NOT_ do, is guarantee the 
stability of the OS, when it is in early development, which is 
where things are today.


>based on what i read, it's only toward the very end of the
>entire testing cycle (FC2-test3) that feature freezes start to
>kick in and we should see things returning to normal in
>preparation for the official release.



More information about the fedora-test-list mailing list