Testing test releases: do not update

shrek-m at gmx.de shrek-m at gmx.de
Sun Feb 22 17:15:38 UTC 2004


Robert P. J. Day wrote:

>against my better judgment, i'm going to take one last stab at explaining
>why i think the current testing procedure is badly defined
>

afair in previous tests:
the schedule was  ~ 1,5 month
no preconfigured *default* update channel

>let's start simple.  as an example of the testing process, red hat
>releases a "test" version -- at the moment, that's FC2-t1.  so far, so
>good.  and red hat obviously wants folks to download, install and beat on 
>that test release and report bugs/errors/issues/whatever.  it all sounds 
>so simple,
>

testing is not a simple act!
it is procedure you have to learn through your experiences and the 
feedback through the  "test-list"  or bugzilla

> but here's where it starts to break down.
>
>guaranteed, within minutes of release, testers will start to find 
>problems.
>  
>

you will read ~ 1000 times the same questions on this TEST-LIST

>some of those bugs could be serious enough
>

who will claim "serious enough" ?
if you never use samba i bet that you are not interested in this bugs

> that they get in the way of further testing,
>

rawhide

> so of course, red hat will quickly release some "updated" RPMs to fix the more serious problems so that testers can get on 
>with the business of testing and find even more problems.
>  
>

rawhide

>so, *knowing* that there will be bugs and, consequently, updated RPMs,
>where can one quickly and conveniently get those RPMs?
>

rawhide

>on the one hand,
>red hat could make everyone painfully and laboriously update packages
>individually, which means 1,000 testers will almost certainly have 1,000
>slightly different test systems -- an obvious nightmare.
>

for test1 and test2 evtl. the best ground for finding sleeping bugs

>or red hat could
>make everyone's life easy (including their own) and give everyone a single
>channel (in yum.conf) against which they could do a simple "yum update".  
>  
>

yum update samba*

>the beauty of that is that 1) it's easy for testers to keep up to date,
>and 2) it guarantees that everyone who uses that channel is running the
>same updated system, so that there's some consistency in bug reports.  
>and what single channel would that be?
>  
>

you will file bugs against
eg.
test1-update1 20040214 10:00
test1-update2 20040214 13:24
test1-update3 20040215 15:23
test1-update4 20040215 15:31
...

and this in this aggressive schedule ?

>as it stands, FC2-t1 was shipped with yum.conf pointing at rawhide.  and 
>what a bad idea that was.  as more than one person has pointed out, 
>rawhide represents the latest, greatest, bleeding edge, 
>not-even-guaranteed-to-work software.
>

this is testing and you make the decission for yourselve what you need 
or what you want

>why on earth would anyone ship a 
>test system which is set up to update against rawhide?
>  
>

you will a nanny-testing ?

>p.s.  i'm just a bit amused by rick johnson's statement that,
>
>  
>
>>>I imagine come FC2 Test 2 or 3, they may split the updates to
>>>updates/test/1.91 or something like that in order to provide us with a
>>>more stable testing ground.
>>>      
>>>
>
>i could have *sworn* that this is the very suggestion i made recently, and 
>it was thoroughly shot down by someone who claimed that it was just 
>waaaaaay too much work for red hat to implement.  apparently not.
>  
>

imho it is not necessary,
"the development is fast and the schedule is aggressive."

it would make more sense with a longer time between test1 test2 test3.

-- 
shrek-m





More information about the fedora-test-list mailing list