spec file changes: removing Release: and %changelog

[Colin Walters]

On Fri, 2008-03-07 at 08:51 -0800, Toshio Kuratomi wrote:

(reordering your reply here)

> I like the ideas for change that you're putting out here but not 
> necessarily the implementations :-).  Things that can be done from 
> within the spec file to affect the buildsystem are vastly preferable to 
> me as our current model is that the srpm is the basic, indivisible unit 
> that we want to work with.
> 
> If you don't see a way to work within that constraint then we as a 
> project need to evaluate whether we want to continue to base ourselves 
> on the srpm or if we're going to move to a new format as our base (with 
> the possibility of generating srpms as an intermediate format).  We 
> might take half-steps like you propose to get there but we need to 
> decide that that's the direction we want to head.

Thanks for the detailed constructive response!  Yes, I agree this is a
crucial point.  The srpm would no longer be the "preferred form of
modification" to use GPL terminology.  

But looking at this realistically, the only point in our current process
(that I know of) where we use srpms internally now is for review of new
contributions.  But we've been talking about a process where we give new
packages tentative revision control space, so they can be imported
without losing history.  So I think the project is already heading in
the direction of using the pkgcvs as the primary/preferred source
format.

So as you say, a key question is - can the goals be achieved without
taking the approach of generating spec files?  Tom suggested a
%{autorelease} macro or the like instead of using spec.in->spec.  I
think that would work in the sense that you could avoid having the spec
file needing to modified by automated processes, but that still means
you need to take the approach of generating a srpm, because vanilla
rpmbuild won't know how to handle %{autorelease} or the like.

I'm a bit torn on this - it makes things slightly easier for just these
modifications, but for more complex changes like importing the package
description from a wiki, or generating the base %files checklist from a
global set based on package tags, we need to take the approach of
generating a spec.

I'll consider this as I continue to prototype out the changes.  The
differences between the macro and spec.in approaches are not large, so
we can switch if necessary.

> Errr... Read the guidelines again.  That is a valid tag due to java 
> packagers having made the case that there is a need to interleave with 
> jpackage.  If the buildsystem were to make the release tag up it would 
> have to decide whether the packager wanted to stay interleavable with 
> jpackage and if so find out what the present jpackage version-release is 
> and build our release tag from that.
> 
> Or you can try to convince the Java Packagers that interleaving with 
> jpackage is not a desirable thing... but we've already been through that 
> once so you better have some good arguments :-)

The only part of the Release tag that I care about the build system
creating is incrementing the number, so that I don't have to spend my
time doing that, and we can automate things more.  

I'm totally fine with allowing the spec file to specify what other bits
go into there.

> >>> * Avoid developer time being spent incrementing integers in spec file
> >> That's usually a 1-character change, sometimes a 2-character change, it takes 
> >> 10 seconds at worst, I fail to see the problem there.
> > 
> > The real thing this blocks is more automation in our processes.  I don't
> > want to have to edit a spec file at all when a new release comes out; we
> > should have a system where I can look on a web page and see "New
> > upstream releases of foo, bar available", and click one link to pull
> > them in for tentative builds and tests, and then if that works tag them.
> > 
> That would be nice.  It would also have to check that the configure 
> script hasn't changed beyond the package's version update (for autoconf 
> using build systems... not sure that other build systems will accurately 
> capture build requirements in one spot :-( ).  Otherwise, you silently 
> go without a new optional feature because configure didn't find it at 
> build time.

Absolutely, it is critical in more automated system that it's easy to
pull down the proposed changes and test things locally too.

You could imagine that when you click on that update link, you get to a
"proposed changes" page where you can see things like the diffs of
configure, number of added/removed files, whether there are new .desktop
files, any new setuid files, etc etc.

> But then you're right back to specifying release information in the spec 
> file :-(.  (It's more elegant in that it's not encoding the build order 
> in the spec but it's still annoying to the packager as they have to add 
> it to the file when things change.) 

Essentially, nothing that we want changed by external processes can be
in the spec file.  If as a first pass we want to allow for automating of
rebuilds, then having the pre/post release information in the spec is
fine because a rebuild won't change that.

However, if we also want to automate pulling in new upstream versions or
SVN/hg/git snapshots, then we need to have the pre/post information in a
separate place.

> Err... it can determine that the last build was a branch-only build but 
> how can it determine that the current build is a branch only build?

I think the simple solution here is to always automatically use the
branch versioning format for non-rawhide builds.

> >> * in schemes like prereleases, how to define the prerelease tag ("alphatag") to 
> >> use.
> > 
> > I like the idea for this one of extending the "sources" file - say a
> > third field which denotes whether a file is a pre or post release.
> > 
> -1
> 
> This is one of the differences between Debian dpkg and rpm that I like 
> about rpm.  All the build information is in one file.  Our current 
> sources file is generated automatically and never needs to be edited by 
> the package builder.  It is only there so the buildsystem can 
> differentiate between two tarballs that have the same name.

What I'm thinking for this is that the sources file would continue to be
generated automatically; instead of doing:

make new-sources FILES=blah-svn20080307.tar.gz
you'd do
make new-sources FILES=blah-svn20080307.tar.gz PRERELEASE=1

(or possibly this is just autodetected by the system; if you add a new
source file whose version would be lower than the existing one, it marks
it as a prerelease)

But really the payoff is longer term, when we can have a package status
web page which can say:

Package foo:
SVN: 47 new changesets [pull]
Releases: 1 new release [pull]

Or even larger scope:

Packages tagged gnome-desktop:

New upstream releases: [pull all]
 gtk+ 2.14.8 [pull]
   - Currently applied patches:
     * http://bugzilla.gnome.org/438221 (fix frobnicator)   [unapply]
     * http://bugzilla.gnome.org/437891 (fix foo)           [unapply]
 gnome-vfs2 2.22.3 [pull]
 ...

> Also, btw, your make local doesn't do the right thing WRT disttag. 
> disttag is supposed to tell which distribution the package is built for, 
> not which repo it is built by.  So disttag == .fc8 means the package was 
> built for and intended to run on Fedora 8.  .local is a repotag rather 
> than a disttag.  Additionally, whatever it expands to should sort lower 
> than the equivalent Fedora package built in the buildsystem.  Otherwise 
> you, the package maintainer, have extra steps to perform to go from 
> testing your local built package to testing the package built on the 
> buildsystem.  (Whereas if it sorts lower, the buildsystem package is 
> picked up by the normal yum update).

Makes sense, yep.  The make local changes are a side thing; I'm mostly
interested in the pkgcvs/koji-related changes.