Metalink support

[Matt Domsch]

On Wed, Mar 19, 2008 at 04:55:35PM +0100, Bram Neijt wrote:
> Yes it sounds like a nice idea and adding the chunk checksums is nice,
> but I doubt anybody on this list would like those checksums to be
> calculated on any of the infrastructure machines at fedora. And
> although I could easily run a service that would generate metalinks
> for you guys, I doubt anybody would like to incorporate it in the
> official site.

no, you wouldn't want to re-generate the checksums.

MM has several components that run on different servers.  The main
turbogears-driven web site, and various helper applications such as
that generate the /publiclist/ web pages, run on a server that has the
master mirror NFS-mounted ro at /pub, so reading files from the file
system, especially small files like the SHA1SUM files, is easy.
 
> Because the SHA1 is already known on the mirror, the updates of
> metalinks can be done very quickly (a script would just do some text
> parsing and posting) and it would allow it to easily create metalinks
> and even keep them up to date (say weekly?).
> 
> Another option is to patch MM, but I think you should also include the
> SHA1 so that would mean you would have to place file metadata in the
> database. If you are going to do that, it might be easier to run
> Bouncer and use the already existing patch for that.
> 
> Simply put: would a solution which runs outside of MM be acceptable?

Depends what "outside of MM" means.  If it's yet another helper script
that can make use of the MM database and local file system available
files, sure, that's fine.  If it's "run outside of Fedora
infrastructure, then copy the data to Fedora infrastructure", no.

Right now, the MM database doesn't have a table listing all the files
anywhere in the file system.  Nor do I want it to grow that if it's
avoidable.  I'm not opposed to adding a table to map Directory, ISO
filename, and checksums, but since we can read and parse the SHA1SUM files
directly, I don't think we'd need to keep that data in the database
too.


-- 
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux