trouble with up2date

Andy Green fedora at
Mon Nov 17 19:26:41 UTC 2003

Hash: SHA1

On Monday 17 November 2003 17:29, Nils Philippsen wrote:

> - You presume that any torrent file must be accessible via http protocol
> which is wrong IMO -- but this could be done with something like
> bittorrent://http://server/path/to/.torrent instead, where http could be
> exchanged for any protocol capable of retrieving files.

Point taken... having two protocols in the URL is a better fit for the 
two-stage torrent/payload retrieval process, it's a clear improvement. 

> - To work properly, BitTorrent relies on the assumption that downloaders
> are uploaders as well. For your scheme everyone would have to download
> and store all packages of a repository on disk. Or even worse, you would
> have to have one torrent for each package which could only be described
> as an "ugly mess".

Idea #1 - "The pseudo-rsync ISO"

Redhat maintain an ISO as usual of the 6-monthly distro versions.  But now in 
addition to keeping that as it is, whenever there is a new release, they 
mount a copy of the original ISO and copy in any updates.  Then they issue a 
1.01 or whatever .torrent for this new super up to date ISO.  People who 
already have the 1.00 version of the CDs suddenly find that their BT download 
for the updated version started at 99%, since BT just needed to update the 
sectors in the iso that were altered by copying in the updated file (BT 
already does this).  Then at the end it renames the .iso to 1.01.  It can be 
mounted at the client to get the individual packages.

If you updated one package, say, then the overhead of fetching the ISO 
directory sector or two that is changed is very small compared to downloading 
just the changed package.  And because most people will be tracking the 
latest ISO, they will be offering it for download too.

Downgrading is just as simple, run BT with the 1.00 .torrent and it will fix 
the 'error' sectors from the newer version back to the old one.

The only downside is the clients need to keep a copy of the isos somewhere, 
but note these are actual usable isos that people download already, not a 
repository mirror.  I have a copy of the Fedora Core 1 ISOs on this machine 
anyway, its not such a crazy idea.

Idea #2 - "Bram we need you"

Dynamic distributed virtual payload generation.  Yes... five buzzwords in one.  
The concept is that there is a master .torrent file which lists the full 
enumeration of all the available packages.... this is exactly like the 
"download everything at once" scenario torrent.

But - uploaders and downloaders can signal to the tracker they have or want a 
subset of the total list of files in the master torrent.  If they actually 
have those files, they are effectively seeders for those parts of the master 
list, if they lack one or more files then they are given the pieces from 
others in the usual BT way.  (Yes this needs some changes in BT and the 
tracker, big job or impossible I have no idea).

Up2date's job would then be to grab the master filelist, assess what packages 
you have and spit out a customized subset .torrent which lists what you 
already have -- so you can contribute to others -- and which new packages you 
want.  Then you hook to the tracker and it connects you to everyone relevant 
to your customized .torrent file, mapping them to the right payload offsets 
for your customized .torrent layout.

- -Andy
Version: GnuPG v1.2.2 (GNU/Linux)


More information about the fedora-list mailing list