Expectation Management for Test Releases

[Chris McDonough]

I'm not sure whether it's clear or not to most people that FC test
releases and/or Rawhide represent unstable software.  It was clear to
me, and I'm certainly no genius.  I can't recall exactly know *how* I
know it, but I do.  So something must have tipped me off at some point. 
Thus, I suspect most other folks know this too (at least subliminally
;-).  I never tested a Redhat beta release before Fedora, so I'm not
sure how the new process differs from the old, though.

But IMHO whether you're talking about a test release or a stable release
of a given piece of software, it's arguably a bit disingenuous of the
person who ultimately desires absolute stability to not contribute
anything back to the testing/development process in the way of code
contribution, bug reports, maillist posts, commiseration, a helping hand
on a config issue, or whatever.  I'd guess that cash even works.

I don't know that your acquaintance at Boeing didn't do some of this,
but you didn't mention that he did.  The fact that he happens to work
for a multinational company and apparently has influence over purchasing
is probably a consideration for Redhat sales, but it might not mean much
to many readers of this list, all of whom are demonstrably trying to
make the software better rather than grousing about how lousy it is
out-of-band (or maybe they're doing both, but at least they are doing
the former too ;-).

With that in mind, I can understand why you'd want to see some
improvements in expectation management for test releases, but maybe it
would be more effective to make the case to Redhat's marketing/sales
department rather than to this list, as they are the folks that are more
likely to have a "dog in that fight".  I'm sure they'd be more than
willing to talk with him and give him a little TLC if he's in a position
of influence.

Personally, I'm pretty amazed at the stability and functionality of
Rawhide.  I expected much, much worse!  It's been rock-solid for me so
far.

On Tue, 2004-04-20 at 13:05, Keith Lofstrom wrote:
> This weekend, at the party following LinuxFest Northwest, I found myself
> across the table from an avionics software project manager at Boeing. 
> He told me of some of his problems with Fedora.
> 
> He said he had big problems running Fedora Core 2 Test 1;  he was not
> mollified when I tried to point out that one can *expect* huge problems
> trying to run test releases.   He insisted that it was entirely
> unprofessional to release software that wouldn't even install, and that
> using the FC2T1 upgrade process on top of a working FC1 system made a
> real hash of it.  He had performed alpha and beta testing on Windows
> and never had such problems. 
> 
> Now, the room was loud, I am hard of hearing, and I may be misrepresenting
> what he said.  He may have meant FC2T2.  I tried to argue that the test
> series was not intended to meet his expectations, and had other purposes.
> I did not manage to make my point.  Perhaps I had a hard time believing
> my own point;  code that doesn't even install DOES look pretty silly. 
> And it is hard to argue about the nature of good code development with
> a person whose career is based on successfully producing some of the
> world's most reliable and mission-critical code.
> 
> This is probably a problem of managing expectations.  It is quite easy
> to navigate from the top of fedora.redhat.com to the FC2Tx download 
> area without encountering a single explicit disclaimer or explanation
> of what FC2Tx is for, or how dangerously buggy the test code can be. 
> Many people, including professional software people heavily oriented
> towards testing, may download this code without understanding what
> they are in for.  You can read this mail group for months without
> seeing a FAQ;  there are apparently none of the warnings that used to
> come with rawhide.  Some big warnings, such as a FAQ that appears often
> on this list, and on the http://fedora.redhat.com/download/test.html
> "download the test release" page, are needed.   We should let potential
> testers know the huge difference between a test release and a general
> availability release,  before they start downloads.  This would save
> this list from a lot of carping from people like the Boeing manager,
> and like me.  State the goals.  Be honest about the downside.
> 
> That will cause a lot of potential testers to go away, sure.  It might
> also prevent disappointments that result in Linux (and Linux programmers)
> being rejected by major software-using organizations.   A decade from
> now, I don't want to ride on an airplane controlled by Windows CE-RT,
> because in 2004 we pissed off the wrong person at Boeing. 
> 
> Let's work on those disclaimers, okay?
> 
> Keith
> 
> -- 
> Keith Lofstrom           keithl at ieee.org         Voice (503)-520-1993
> KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
> Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs
>