trouble with up2date
fedora at warmcat.com
Mon Nov 17 19:26:41 UTC 2003
-----BEGIN PGP SIGNED MESSAGE-----
On Monday 17 November 2003 17:29, Nils Philippsen wrote:
> - You presume that any torrent file must be accessible via http protocol
> which is wrong IMO -- but this could be done with something like
> bittorrent://http://server/path/to/.torrent instead, where http could be
> exchanged for any protocol capable of retrieving files.
Point taken... having two protocols in the URL is a better fit for the
two-stage torrent/payload retrieval process, it's a clear improvement.
> - To work properly, BitTorrent relies on the assumption that downloaders
> are uploaders as well. For your scheme everyone would have to download
> and store all packages of a repository on disk. Or even worse, you would
> have to have one torrent for each package which could only be described
> as an "ugly mess".
Idea #1 - "The pseudo-rsync ISO"
Redhat maintain an ISO as usual of the 6-monthly distro versions. But now in
addition to keeping that as it is, whenever there is a new release, they
mount a copy of the original ISO and copy in any updates. Then they issue a
1.01 or whatever .torrent for this new super up to date ISO. People who
already have the 1.00 version of the CDs suddenly find that their BT download
for the updated version started at 99%, since BT just needed to update the
sectors in the iso that were altered by copying in the updated file (BT
already does this). Then at the end it renames the .iso to 1.01. It can be
mounted at the client to get the individual packages.
If you updated one package, say, then the overhead of fetching the ISO
directory sector or two that is changed is very small compared to downloading
just the changed package. And because most people will be tracking the
latest ISO, they will be offering it for download too.
Downgrading is just as simple, run BT with the 1.00 .torrent and it will fix
the 'error' sectors from the newer version back to the old one.
The only downside is the clients need to keep a copy of the isos somewhere,
but note these are actual usable isos that people download already, not a
repository mirror. I have a copy of the Fedora Core 1 ISOs on this machine
anyway, its not such a crazy idea.
Idea #2 - "Bram we need you"
Dynamic distributed virtual payload generation. Yes... five buzzwords in one.
The concept is that there is a master .torrent file which lists the full
enumeration of all the available packages.... this is exactly like the
"download everything at once" scenario torrent.
But - uploaders and downloaders can signal to the tracker they have or want a
subset of the total list of files in the master torrent. If they actually
have those files, they are effectively seeders for those parts of the master
list, if they lack one or more files then they are given the pieces from
others in the usual BT way. (Yes this needs some changes in BT and the
tracker, big job or impossible I have no idea).
Up2date's job would then be to grab the master filelist, assess what packages
you have and spit out a customized subset .torrent which lists what you
already have -- so you can contribute to others -- and which new packages you
want. Then you hook to the tracker and it connects you to everyone relevant
to your customized .torrent file, mapping them to the right payload offsets
for your customized .torrent layout.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
-----END PGP SIGNATURE-----
More information about the fedora-list