Source URL guidelines (was Re: source file audit - 2009-02-15)

[Michael Schwendt]

On Sun, 22 Feb 2009 07:37:28 +0100, Ralf wrote:

> The whole point behind Source-URL rules is to have a reliable, 

Making a Source-URL reliable is not the package maintainer's
responsibility. All that matters is that the URL works during a package
review request and at least does not give a 40x error. As some upstream
projects like to change their web page directory structure from time to
time, it can happen that download locations change, too. Rebuilding
tarballs is done by some projects, too, for minor/subtle fixes even
in readme files.

> deterministic URL from which a package can be retrieved from for e.g. 
> verification (e.g checksum), legal reviews, tracking origins of packages 
> etc.

How often that does happen?

There still is the URL tag which can be used to search for [and verify!]
new download locations during a "legal review".

> and to prevent Fedora from being vulnerable from upstream dynamics 
> (low quality random snapshots, bugs, compromised upstreams, etc.)

?!  A static Source-URL alone doesn't achieve that alone. I see value in
re-downloading tarballs regularly in order to verify checksums, but that
doesn't protect against "low quality random snapshots, bugs, compromised
upstreams". It can happen that a tarball has been compromised already when
the packager downloads it (mind you, we advise upstream devs to use
detached GPG signatures). Only if upstream becomes aware of it and
updates/removes the tarball, the Source-URL checker can notice it. The
checker also doesn't know whether a tarball is out-of-date, bug-infested,
vulnerable, since updates may have been published in the same or a
different directory already.

> That said, the sourceforge rule is a "best practice's hint" to _prevent_ 
> users from populating source-urls with one of sourceforge's mirror.

Historically, its goal has been a different one:

Avoiding that packagers point to the interactive mirror-selection web page
at SF.net. Reviewers [still] prefer wget/curl-compatible download locations,
although they need to verify the home page and download location manually
anyway.

> <cite>
> For packages hosted on sourceforge, use
> 
> Source0: http://downloads.sourceforge.net/%{name}/%{name}-%{version}.tar.gz
> 
> changing ".tar.gz" to whatever matches the upstream distribution. Note 
> that we are using downloads.sourceforge.net instead of an arbitrarily 
> chosen mirror.
> </cite>

This has been found to "work most of the time" (while older ones like
dl.sf.net stopped being reliable), but it's not bullet-proof either.
It can happen that you're pointed at a mirror that cannot be connected due
to timeouts - the direct url to a specific/hardcoded mirror is just
fine, or else a packager would waste time on getting urls right instead
of spending time on more important matters. And some projects store their
files in their web space instead of the sf.net download system.