The Future of Fedora Package Management and the RPM Philosophy

[Jeffrey C. Ollie]

It's great to see discussion about the future Fedora package SCM again.
However, I think that some people (myself included) are getting a little
bogged down in the details of how to manage branches in a source code
repository or the details of how particular SCMs work. Before deciding
on details like this, there's one big question that we need to answer
before we can make intelligent decisions about SCMs and how we should
use them.

The question is, do we want to move away from RPM's philosophy of using
pristine sources plus patches to build binary packages? Determining the
answer to this question is a fundamental first step before decide
“what's next.”

Currently, all of the packages in Fedora consist of pristine sources,
patches to the pristine sources, and a spec file that contains
instructions that tell the builders how to take the pristine source,
apply the patches, and output a binary package. All of this package
building information is managed in a source code repository. The build
system pulls the information out of the package repository to build the
binary RPMs. This process has served the Fedora Project well so far. The
packages in Fedora, thanks to a lot of hard work by packager maintainers
and reviewers – guided by the packaging guidelines – are already very
high quality and getting better all of the time.

What hasn't been done well so far is the management of the patches that
we apply to the sources. As far as the package repository is concerned
patches appear out of thin air. This is because patches have not been
managed in a central and public manner by the Fedora project (other than
storing them as blobs in the package repository). Package maintainers
have had to devise their own systems for developing and managing
patches, and this has meant that the patches have been developed in an
ad-hoc and private manner.

This has made life difficult in a number of ways. It's harder for
package maintainers to manage patches, both on their own and in
collaboration with other package maintainers. It's harder to communicate
changes to upstream and downstream developers.

Managing patches in a central and public manner will have the following
benefits:

     1. Make the packager maintainer's work easier by making it easier
        to forward-port patches as new releases of upstream code become
        available or to backport bug fixes and security patches as they
        are developed. It will also be easier for package maintainers to
        collaborate on patches. Making the the task of managing patches
        easier will indirectly improve the quality of the packages
        because it will be easier to update packages when new sources
        are released upstream or to backport bug or security fixes that
        have been committed to the upstream SCM but are not yet part of
        a formal release.
        
     2. Make it easier to communicate changes to upstream developers (so
        that patches that fix bugs or add features can benefit the wider
        F/OSS community and the package maintainer doesn't need to
        maintain the patches indefinitely). Fairly or unfairly, there's
        the perception that many patches sit in our CVS and never get
        pushed upstream. Of course, some patches should never get pushed
        upstream since they represent Fedora-specific policy, but in
        most cases we'd all be better off if our patches were
        incorporated upstream.
        
     3. Make it easier for downstream developers (e.g. the RHEL and OLPC
        engineers or anyone else that repackages Fedora) to add their
        own customizations and communicate those changes back to Fedora
        or the upstream developers. Of particular concern here would be
        making it easier for downstream distributions to apply patches
        that implement policies specific to those distributions while
        keeping it easy for them to track changes in the Fedora patches.

There are (at least) two different approaches that we could take to
managing patches in a central, public manner. One method would keep the
traditional package repositories and add separate patch management
repositories. This method would preserve the pristine source plus
patches philosophy of RPM. Another more radical approach would be to
integrate the package and patch management repositories into one.

In the separate package and patch management repositories, the package
management remains largely the same. The SCM technology used to maintain
the package repository might change, but it would remain a collection of
pristine source, patches, and spec files.

The patch management repository for a package would be different – it
would consist of a “vendor” branch that contained the unmodified
upstream code and it would contain a number of “patch” branches that
represent the patches to the upstream code that appear in the source
package repository. Ultimately, every patch that appears in the package
repository would have a branch in the patch management repository.

Since the development of patches now happens in a central, public manner
it's easier to communicate changes upstream, downstream, and within the
Fedora community. Doing things in a central, public manner means that we
can develop tools and procedures that will make managing patches easier
for the package maintainer.

If we want to be more radical, we could integrate the package and the
patch repositories. Package building would no longer use pristine
sources and patches to produce a binary package. Instead, the build
system would pull already-patched code out of the repository and build
the binary package from there.

The advantage to integrating the patch and package repositories would be
reducing the package maintainer's work. With separate package
repositories and patch repositories the package maintainer has to do
some work to export patches from the patch repository and import them
into the package repository (I believe that we could develop some tools
to minimize the amount of work it takes to export/import patches, but
there would always be some manual steps).

The primary disadvantage to integrating patch and package management is
that we move away from RPM's philosophy of using pristine sources.
Pristine sources have been required because it's possible to easily
verify that our copy of the sources matches the upstream copy by
comparing MD5 or GPG signatures. With an integrated repository where
there are no longer pristine sources, it's not possible to verify that
our copy of the code matches the upstream copy by comparing signatures
on tarballs. Verifying the code integrity is possible with the
integrated repository, but it's potentially much more difficult.

Another disadvantage of integrated management is that unless the package
maintainer disciplines himself/herself it can become difficult to
separate changes to the code that were done to implement Fedora-specific
policies (changing defaults in configuration files, moving files around
to suit the FHS, etc.) from changes that were done to fix bugs or
security problems (and thus might be of interest to upstream
developers). Guidelines on how to manage vendor, patch, and policy
branches will help maintain that discipline (but vigilance will be
necessary).

So would moving to an integrated package and patch repository be worth
it? It's hard to say. Some of the predecessors of RPM used modified
sources to build packages – one of the complaints about those early
package management systems was that it was hard to keep track of local
changes to the code. However, with the advantage of modern source code
management systems keeping track of changes to the code shouldn't be an
issue.

In either case we need to get the development and maintenance of patches
“out in the open” and that means having patches developed in central,
public repositories.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20070608/50579d6a/attachment.sig>