[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Testing test releases: do not update

Robert P. J. Day wrote:

against my better judgment, i'm going to take one last stab at explaining
why i think the current testing procedure is badly defined

afair in previous tests: the schedule was ~ 1,5 month no preconfigured *default* update channel

let's start simple. as an example of the testing process, red hat
releases a "test" version -- at the moment, that's FC2-t1. so far, so
good. and red hat obviously wants folks to download, install and beat on that test release and report bugs/errors/issues/whatever. it all sounds so simple,

testing is not a simple act!
it is procedure you have to learn through your experiences and the feedback through the "test-list" or bugzilla

but here's where it starts to break down.

guaranteed, within minutes of release, testers will start to find problems.

you will read ~ 1000 times the same questions on this TEST-LIST

some of those bugs could be serious enough

who will claim "serious enough" ? if you never use samba i bet that you are not interested in this bugs

that they get in the way of further testing,


so of course, red hat will quickly release some "updated" RPMs to fix the more serious problems so that testers can get on with the business of testing and find even more problems.


so, *knowing* that there will be bugs and, consequently, updated RPMs,
where can one quickly and conveniently get those RPMs?


on the one hand,
red hat could make everyone painfully and laboriously update packages
individually, which means 1,000 testers will almost certainly have 1,000
slightly different test systems -- an obvious nightmare.

for test1 and test2 evtl. the best ground for finding sleeping bugs

or red hat could
make everyone's life easy (including their own) and give everyone a single
channel (in yum.conf) against which they could do a simple "yum update".

yum update samba*

the beauty of that is that 1) it's easy for testers to keep up to date,
and 2) it guarantees that everyone who uses that channel is running the
same updated system, so that there's some consistency in bug reports. and what single channel would that be?

you will file bugs against eg. test1-update1 20040214 10:00 test1-update2 20040214 13:24 test1-update3 20040215 15:23 test1-update4 20040215 15:31 ...

and this in this aggressive schedule ?

as it stands, FC2-t1 was shipped with yum.conf pointing at rawhide. and what a bad idea that was. as more than one person has pointed out, rawhide represents the latest, greatest, bleeding edge, not-even-guaranteed-to-work software.

this is testing and you make the decission for yourselve what you need or what you want

why on earth would anyone ship a test system which is set up to update against rawhide?

you will a nanny-testing ?

p.s. i'm just a bit amused by rick johnson's statement that,

I imagine come FC2 Test 2 or 3, they may split the updates to
updates/test/1.91 or something like that in order to provide us with a
more stable testing ground.

i could have *sworn* that this is the very suggestion i made recently, and it was thoroughly shot down by someone who claimed that it was just waaaaaay too much work for red hat to implement. apparently not.

imho it is not necessary, "the development is fast and the schedule is aggressive."

it would make more sense with a longer time between test1 test2 test3.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]