Testing Fedora - small (?) suggestion.

[Gilboa Davara]

On Fri, 2006-11-10 at 18:19 -0300, Horst H. von Brand wrote:
> Gilboa Davara <gilboad at gmail.com> wrote:
> > I use Fedora daily, and I have a vested interest in -actively- helping
> > Fedora (and in-turn RHEL) become a better product.
> 
> Report problems as you find them. Check out test packages for stuff you
> care about.

Being done.

> > I don't have a dedicated Rawhide machine, I do play with it from time to
> > time using VMWare Server, but this is fairly light testing as I'm pretty
> > short on time. (Just to see what's new)
> > I -do- try to install (at least on a VM machine) every test release and
> > give it a day or two of testing - but even this is hardly enough.
> 
> It will help weed out DOA problems, so it does help.

I know.
That's why I'm doing it ;)

> 
> > FC6 was a good release, but certain managed to slip through... some of
> > them rather critical (i586 kernel springs to mind) which should have
> > been found by us. (Read: testers)
> 
> Need more of that stuff. Right.
> 
> > As I see it, there's not enough Rawhide users to reduce the release bug
> > count, mainly because:
> > A. In my experience, 1 out of 2 rawhide installs break due to missing
> > dependencies - and spending four hours (and ~1+GB of bandwidth) just to
> > watch Anaconda choke on missing pygtk package is very frustrating.
> 
> I'm running rawhide, udated (almost) daily by yum. When stuff doesn't
> update OK, it is usually enough to add it with --exclude and try
> again. Luckily, yum keeps what it's got already in cache, so...

Problem is, using yum to install Rawhide (from latest stable release)
doesn't test the installer, and the installer is one of the biggest test
items.
(... Touchy subject, I just lost a machine due to a botched upgrade...
eeek! [already in Bugzilla])

> 
> > B. Rawhide tends to break, a-lot.
> 
> Not in my experience... yes, packages do break (sometimes), but on the
> whole it works just fine (for me).

During the FC4 -> FC5 -> FC6. I tried installing (from scratch) and/or
upgrading (using Anaconda) Rawhide at least 10 times.
If I recall, I only see firstboot... err.... twice.
That's ~20% success rate.

Doing yum -> rawhide is easy... but as I said, Anaconda is my favorite
testing pet ;)

> 
> > C. As much as I disagree with this notion, people view Fedora as RHEL's
> > Alpha and view Rawhide as "experimental Alpha".
> 
> Can't change that perception easily...

But we -should- try.

> 
> > More then anything FC6 proved that 3 test releases may not be enough to
> > get a bug free release,
> 
> A 100 test releases won't, so...

I agree.

> 
> >                         so let me suggest the following:
> > A. Create more mile-stone releases. Once the tree reaches build
> > integrity (no missing packages), spin a test release. (Fixes P1, P2)
> 
> Test 1 should (at the very least) be there, and so should the others.

OK.

> > B. Change the terms that are being used to describe each test release.
> > Whether we like it or not, people are used to the "Alpha", "Beta" and
> > "RC" terms, and tend to consider "Test release" as "Alpha release". I
> > understand that the term "Test" was used to differentiate the
> > ever-rolling Fedora from the release-based RHEL, but Fedora has aged
> > enough to be viewed as an entity by itself and we can drop the "Test"
> > term.
> 
> Yep. 
> 
> Alpha == don't ever use it if you aren't downright suicidal
> Beta  == Use only when you don't care one bit for your data/machine/sanity
> RC    == Let's wait until it is there "for real", let others get burned

People who view Alpha/beta/RC as suicidal won't touch test release and
won't even think about using Rawhide, so we don't really care about them
(in the testing context), do we?

As I see it, most people won't touch Alpha, but some brave souls will.
(Lets assume Test1 is pretty much Alpha-stuff)
More people will touch Beta 1 - a number that will slowly increase as
Beta count progresses.
When the thing hits RC, usage numbers should increase dramatically.
(look at the Vista download count... they are huge! [and we are talking
about ... Windows users. *Cough]).

RC releases give you enough time to fix last minute bugs that may make
Fedora look like Alpha-ware once the main release is out. (Read: missing
print_tained export [FC5] or i586 kernel [FC6]). 

> 
> > Using "normal" terms should help combat the "RHEL Alpha" and 'RHEL Alpha
> > experimental" image problem (Fixes P3). 
> 
> Fedora users /should/ have gotten the message by now...

Yes, but Fedora testers < Fedora users < Linux users < Software users.
I (as in, well, I) want a bigger share of the pie.
More users and in the long term, more testers.

> 
> > C. Once Fedora hits RC, only bug fixes go into the tree. No last minute
> > 2.6.39 kernel that break Anaconda, SCSI, and USB two days before the
> > release. Nada. New features can always enter the tree as updates once
> > the release ISOs have been sent.
> 
> Look at the roadmaps that have been published for, e.g., Fedora 5 and 6...

I don't remember much about FC1/2/3, but FC4/5/6 all over-shoot the
initial schedule - FC5 and 6 saw lot of last minute breakage.
(Though I'm the last to talk about schedules... the project I'm working
on is 8 months over-due... ;))

- Gilboa