Testing Fedora - small (?) suggestion.

[Gilboa Davara]

On Sat, 2006-11-11 at 12:36 -0900, Jeff Spaleta wrote:
> On 11/11/06, Gilboa Davara <gilboad at gmail.com> wrote:
> > On Sat, 2006-11-11 at 00:28 -0900, Jeff Spaleta wrote:
> > > On 11/10/06, Gilboa Davara <gilboad at gmail.com> wrote:
> > > > A. Create more mile-stone releases. Once the tree reaches build
> > > > integrity (no missing packages), spin a test release. (Fixes P1, P2)
> > >
> > > Logic fault... we dont have enough testers right now... more
> > > mile-stone releases will actually mean even fewer people will be
> > > testing each of those releases.  We don't magically gain more manhours
> > > of testing by having 14 individual isosets in the wild.
> >
> >
> > You've missed my point.
> > Take Mozilla - each and every night they release a nightly build.
> 
> First of all you can't respin all of rawhide in one night. Are you
> seriously suggesting that we re-engineer rawhide in such a way that we
> do a full respin of the entire codebase and then push the full tree as
> one unit..everytime we make a public push?  Let's keep the scale of
> the problems in mind please.  Rawhide has dependancy issues because
> the codebase is so large that it can't be easily re-mastered in one
> big push. Mass rebuilds when they are required are major multiday
> efforts, and we can not rely on that left of churn as the primary way
> to get individual component updates out and tested.

OK. Bad selection of example on my part.
It was not my intention to get a nightly ISO build.
I am looking for a way to get a 1/1 install/boot ratio, which means,
unless something is broken within Anaconda itself, for each
install/upgrade attempt, I'd be getting a bootable image.

> 
> > Point of reason:
> > A. The biggest issue with any Linux distribution is the installer.
> 
> I'm going to focus on this one item, since most of the others are
> associated with the idea of getting better testing of the installer
> specifically. I have an alternative proposal for installer testing
> instead of attempting to force more structure into the rawhide
> process.  I agree that the installer itself is a very special case, a
> case that rawhide's structure isn't well suited for.
> 
> If we want to develop a regiment for testing the installer
> functionality specifically... then we should be discussing nightly
> builds of the installer against a simple to use and very static
> package-payload that does not represent the full rawhide image.  We
> don't need full rawhide tree consistency to better test the installer.
> 
> There's no reason that we need to make the full rawhide tree available
> for people who want to continually test the installer functionality
> and do regression testing for things like partition creation and
> handling. All you need to do is to identify a basic set of payload
> packages that is neceessary to test the installer.. make that payload
> set semi-static.. and pump out test images of the installer as the
> installer codebase gets updated using this cutdown package payload.
> Update that cutdown package payload as it become self-consistent
> regardless of the full state of the rawhide tree.  We don't need
> openoffice to be in a consistent state in the full rawhide tree to
> test the installer for regressions.
> 
> So let me recap for anyone who could make this happen.
> 
> Problem statement:
> Better testing of the installer functionality regardless of the state
> of the full rawhide tree.
> 
> Proposal:
> creation of a semi-static payload tree that nightly builds of the
> installer targets so testers can avoid depchain breakage at install
> time, while still being able to test all installer based
> functionality.  Additionally the semi-static payload would provide a
> test of the firstboot process as well.  While there is no substitute
> for bare metal, allow for a xen based testing process so testers
> without dedicated hardware can chew on these nightly installer builds
> to do some regression testing.
> 
> Thoughts from the release-eng and installer people inside RedHat as to
> the feasibility and usefulness of this proposal?
> 
> -jef


Good idea, but no go.
If you create a static test case, upgrade can no longer be tested.
In recent years I rarely seen a non-Rawhide Anaconda blowing up during
fresh install.
Anaconda/FC6 alone killed two upgrades.

Gilboa "Just fresh-installed his laptop after a botched FC6 upgrade"
Davara.