Testing test releases: do not update
shrek-m at gmx.de
shrek-m at gmx.de
Sun Feb 22 17:15:38 UTC 2004
Robert P. J. Day wrote:
>against my better judgment, i'm going to take one last stab at explaining
>why i think the current testing procedure is badly defined
>
afair in previous tests:
the schedule was ~ 1,5 month
no preconfigured *default* update channel
>let's start simple. as an example of the testing process, red hat
>releases a "test" version -- at the moment, that's FC2-t1. so far, so
>good. and red hat obviously wants folks to download, install and beat on
>that test release and report bugs/errors/issues/whatever. it all sounds
>so simple,
>
testing is not a simple act!
it is procedure you have to learn through your experiences and the
feedback through the "test-list" or bugzilla
> but here's where it starts to break down.
>
>guaranteed, within minutes of release, testers will start to find
>problems.
>
>
you will read ~ 1000 times the same questions on this TEST-LIST
>some of those bugs could be serious enough
>
who will claim "serious enough" ?
if you never use samba i bet that you are not interested in this bugs
> that they get in the way of further testing,
>
rawhide
> so of course, red hat will quickly release some "updated" RPMs to fix the more serious problems so that testers can get on
>with the business of testing and find even more problems.
>
>
rawhide
>so, *knowing* that there will be bugs and, consequently, updated RPMs,
>where can one quickly and conveniently get those RPMs?
>
rawhide
>on the one hand,
>red hat could make everyone painfully and laboriously update packages
>individually, which means 1,000 testers will almost certainly have 1,000
>slightly different test systems -- an obvious nightmare.
>
for test1 and test2 evtl. the best ground for finding sleeping bugs
>or red hat could
>make everyone's life easy (including their own) and give everyone a single
>channel (in yum.conf) against which they could do a simple "yum update".
>
>
yum update samba*
>the beauty of that is that 1) it's easy for testers to keep up to date,
>and 2) it guarantees that everyone who uses that channel is running the
>same updated system, so that there's some consistency in bug reports.
>and what single channel would that be?
>
>
you will file bugs against
eg.
test1-update1 20040214 10:00
test1-update2 20040214 13:24
test1-update3 20040215 15:23
test1-update4 20040215 15:31
...
and this in this aggressive schedule ?
>as it stands, FC2-t1 was shipped with yum.conf pointing at rawhide. and
>what a bad idea that was. as more than one person has pointed out,
>rawhide represents the latest, greatest, bleeding edge,
>not-even-guaranteed-to-work software.
>
this is testing and you make the decission for yourselve what you need
or what you want
>why on earth would anyone ship a
>test system which is set up to update against rawhide?
>
>
you will a nanny-testing ?
>p.s. i'm just a bit amused by rick johnson's statement that,
>
>
>
>>>I imagine come FC2 Test 2 or 3, they may split the updates to
>>>updates/test/1.91 or something like that in order to provide us with a
>>>more stable testing ground.
>>>
>>>
>
>i could have *sworn* that this is the very suggestion i made recently, and
>it was thoroughly shot down by someone who claimed that it was just
>waaaaaay too much work for red hat to implement. apparently not.
>
>
imho it is not necessary,
"the development is fast and the schedule is aggressive."
it would make more sense with a longer time between test1 test2 test3.
--
shrek-m
More information about the fedora-test-list
mailing list