Testing Fedora - small (?) suggestion.

[Jeff Spaleta]

On 11/11/06, Gilboa Davara <gilboad at gmail.com> wrote:
> On Sat, 2006-11-11 at 00:28 -0900, Jeff Spaleta wrote:
> > On 11/10/06, Gilboa Davara <gilboad at gmail.com> wrote:
> > > A. Create more mile-stone releases. Once the tree reaches build
> > > integrity (no missing packages), spin a test release. (Fixes P1, P2)
> >
> > Logic fault... we dont have enough testers right now... more
> > mile-stone releases will actually mean even fewer people will be
> > testing each of those releases.  We don't magically gain more manhours
> > of testing by having 14 individual isosets in the wild.
>
>
> You've missed my point.
> Take Mozilla - each and every night they release a nightly build.

First of all you can't respin all of rawhide in one night. Are you
seriously suggesting that we re-engineer rawhide in such a way that we
do a full respin of the entire codebase and then push the full tree as
one unit..everytime we make a public push?  Let's keep the scale of
the problems in mind please.  Rawhide has dependancy issues because
the codebase is so large that it can't be easily re-mastered in one
big push. Mass rebuilds when they are required are major multiday
efforts, and we can not rely on that left of churn as the primary way
to get individual component updates out and tested.

> Point of reason:
> A. The biggest issue with any Linux distribution is the installer.

I'm going to focus on this one item, since most of the others are
associated with the idea of getting better testing of the installer
specifically. I have an alternative proposal for installer testing
instead of attempting to force more structure into the rawhide
process.  I agree that the installer itself is a very special case, a
case that rawhide's structure isn't well suited for.

If we want to develop a regiment for testing the installer
functionality specifically... then we should be discussing nightly
builds of the installer against a simple to use and very static
package-payload that does not represent the full rawhide image.  We
don't need full rawhide tree consistency to better test the installer.

There's no reason that we need to make the full rawhide tree available
for people who want to continually test the installer functionality
and do regression testing for things like partition creation and
handling. All you need to do is to identify a basic set of payload
packages that is neceessary to test the installer.. make that payload
set semi-static.. and pump out test images of the installer as the
installer codebase gets updated using this cutdown package payload.
Update that cutdown package payload as it become self-consistent
regardless of the full state of the rawhide tree.  We don't need
openoffice to be in a consistent state in the full rawhide tree to
test the installer for regressions.

So let me recap for anyone who could make this happen.

Problem statement:
Better testing of the installer functionality regardless of the state
of the full rawhide tree.

Proposal:
creation of a semi-static payload tree that nightly builds of the
installer targets so testers can avoid depchain breakage at install
time, while still being able to test all installer based
functionality.  Additionally the semi-static payload would provide a
test of the firstboot process as well.  While there is no substitute
for bare metal, allow for a xen based testing process so testers
without dedicated hardware can chew on these nightly installer builds
to do some regression testing.

Thoughts from the release-eng and installer people inside RedHat as to
the feasibility and usefulness of this proposal?

-jef