package updates for in all repos at the same time (Was: Incoming: directfb soname problems)

[Thorsten Leemhuis]

Am Sonntag, den 14.05.2006, 16:19 +0200 schrieb Michael Schwendt:
> On Sun, 14 May 2006 13:50:13 +0200, Thorsten Leemhuis wrote:
> 
> > > Remember a few things:
> > > 
> > > 1) A regular [updates-]testing repository exposes only some users to
> > > package releases which may or may not become official. This adds another
> > > upgrade path. Do we work around bugs introduced in these updates? 
> > What kind of bugs do you mean exactly? How does core handle it?
> For instance: A bug in a package scriptlet, which touches some files and
> damages them. Will the final update try to repair the damage?

Good question -- I think that needs to be handled in a case by case
decision. And it nothing that hopefully happens really that often.

> Or: Accidentally dropped features, which are missing because a configure
> script failed to detect upgraded build requirements and hence disabled
> some pieces. In several applications, related configuration settings are
> lost when features are removed (a concrete example is Sylpheed).
> 
> One thing for sure, both are scenarios which can happen.

Agreed. And that's why it we IMHO should have a testing repo to protect
the users a bit more than we do now ;-)

>  However, from a
> user's perspective it must be clear in which ways another extras-testing
> repository differs from the normal extras repository.

No real difference -- just a "they hang out there some days before they
hit the usual repo". 

> Increased risk -> possibly increased requirement to recover from broken
> test updates -> the smell of guinea pigs -> the optional testing
> repository does not become popular -> doubtful benefit.

Depends on how it is used. There shouldn't be and increased risk with my
planed way to run it (just a repo where updated packages Fifo before
they hit the proper repo) 

> > > Do
> > > enough active users enable such an optional repository, so that it would
> > > become really helpful? 
> > That's always problematic. But that's not a reason not to do it. It IMHO
> > would be helpful enough if only all the Extras packagers would enable it
> > and test their own packages falling out plague. (they can do that with
> > the needsign-repo, too, but I doubt that a lot of people do that)
> Most likely because there is no convenient way to pull a package from the
> needsign-repo [and any deps built against it]. And no way to put a lock on
> a set of packages in the needsign queue in order to release the complete
> set as a whole.

Yes, that could lead to problems -- did we hit such cases yet? (that's
not meant as a rhetoric question -- more it "if the answer is 'yes,
often'  then we might need to do something to avoid this in the future)

> How about picking up the old idea of a "scratch target" instead?

The idea is still around -- I think there are mainly two things blocking
it
- update to new plague 
- someone that's works out the details how it actually works and that
drives the whole idea forward

> > > I think Fedora Core Test Updates suffer from quite
> > > some desinterest.
> > Well, it's not perfect, but I think it works good enough. 
> As a counter-example, createrepo for FC4 and FC3 has still not been
> updated to fix the broken deps in Extras' "plague". If the Test Update
> repository were popular and helpful and if the developers believed in it,
> the errata for FC4/FC3 would have happened already long ago.

The Core testing suffers IMHO from this: updates are posted there for
testing and get forgotten. That's why I proposed a time-based scheme.

> > >  Lack of feedback--also success reports--about test
> > > updates doesn't help you to decide whether an update is good.
> > Well, currently there is no testing or decision if a update is good at
> > all (besides the checking from the packager himself). As a ordinary user
> > I'd much more prefer a package that was in extras-testing for some days
> > (without feedback) over one that got no testing at all.
> ?  Is a package in extras/5, which is four weeks old, more safe to use
> than a package in extras/5, which is two hours old? 

Partly -- because the older package probably is in use on several
machines already; if it would have major problems people would have
filed bug-reports and we would have removed or fixed the package in
between. For the one that is two hours old it is likely that I'm one of
its first users -- I'm the tester then, if I want or not.

Example: A package which is in extras/5 for two hours after it was in
extras/testing/5 for a weeks IMHO is more safe to use then a package
that is build three hours ago and was published directly to the repo
(even the package maintainer might be sleeping and couldn't test the
package due to that yet)

> The age of a package
> is irrelevant. The number of competent (!) testers who actually tested it
> and report success or failure, is the interesting bit.

Sure. But currently we have no tester's at all. I'd like to have
testers. 

> > And to make it sure (in case it wasn't already): No, I don't want a
> > explicit testing repo where stuff could be published and tested that is
> > not ready yet. I only want a testing repo where all updates hang out
> > some days before they hit the proper repo. First in and First out (if
> > there is no major breakage).
> And *if* there is major breakage? Which pieces would you want to hold up?
> And how?

That is outside the scope of this particular discussion afaics -- sure,
the questions need answers. But a "major breakage" can happen any time
-- with or without testing repo is mostly irrelevant. But a testing repo
might limit the impact because only those that have testing enabled will
be affected and not everyone.

> > >  And how are test updates pushed
> > Fifo. Example: Package foo is pushed to testing today (Sunday) and bar
> > on Monday. foo will be pushed to the proper repo on Friday and bar on
> > Saturday.
> Before that becomes possible we will need a way to mark packages which
> depend on eachother, so they are only pushed together.

We probably need that, yes. But we simply could use the current scheme
until then: packages foo1 and foo2 are build and a particular point in
time and pushed to the testing repo together and another point of time.
bar1 and bar2 get build some days later and also pushed to testing
together. Five days after foo1 and foo2 were pushed both pacakgers are
moved from testing to the proper repo. bar1 and bar2 would follow
together five days after their push.

>  Plus a way to
> prioritise build jobs and releases (not just with security fixes in mind).

Well, we need that in any case. 

> > > or withdrawn when dependencies have or have not been built against them?
> > good question. But that shouldn't normally be the case. 
> > Another example (in case a problem showed up): Package foo is pushed to
> > testing today (Sunday) and bar on Monday (bar depends on the new foo).
> > foo makes problems. foo is fixed quickly on Tuesday. foo and bar get
> > pushed to the proper repo on Saturday.
> Automatically?

Unsure. But yeah, why not.

>  So, Extras packagers must watch another moving target?

What do they have to watch precisely? I only seem problems when some
packages are pushed directly to the repo. That should only happen for
security issues -- we IMHo should be able to handle that.

> > > How does an additional repo solve the problem of insufficient
> > > communication about upgrades and rebuilds of complete dependency chains?
> > Not at all. That's not it's purpose. /me feels misunderstood 
> Then I must admit I don't understand the purpose of an extras-testing
> repository (in the context of how we've been doing builds/releases for
> quite some time).

Seems so.

> > /me sometimes doesn't understand the Open-Source-World -- a lot of
> > people (see the achieves of the fedora mailinglists) scream when Fedora
> > Core updates a package directly without letting it linger around in
> > updates-testing for some days.
> 
> ?
> 
> Well, I see no users who scream and demand an extras-testing repository.

/me is one user and I'd like to have one ;-) Others around?

But it really seems I'm in the minority in Extras land. So I'll be quiet
for another while again and will probably stay away from this
discussion.

> We must improve in the area of _Package Upgrade Guidelines/Policies_,

Agreed.

> not create extra burden and artificial hurdles for the packagers.

I don't see any new burden for the packagers. 

CU
thl
-- 
Thorsten Leemhuis <fedora at leemhuis.info>