Jigdo - A Professional Letter to Mike McGrath

[Francois Petillon]

Jonathan Steffan wrote:
> * Any Spin: Not all mirrors chose to carry the ISO images. A next-hop
> or local mirror might not be available with the ISO images for direct
> download. This very close mirror will be able to be used to "put
> together" the ISO image and with our awesome new MirrorManager the
> acquisition of this data source will be automatic.
> * Bandwidth Optimization/Utilization: We are able to utilize mirrors
> around the globe without requiring mirror admins to think twice about
> hosting Jigdo data, they already are hosting most of the data needed to
> put the image back together and have to install no additional software
> (as in the case of running a torrent seed.)

I will speak as a mirror admin (ftp.free.fr/ftp.proxad.net). As far as I 
am concerned, bandwidth is not a real issue but disk IOs are. Disk 
capacity is growing exponentially, sequential access bandwidth grows 
linearly but disk seeks decrease at a very slow pace (SSD are coming but 
they do not match yet the needed capacity). Thus, improving server 
performances means either adding more disks or optimizing disks access 
(ie reading more data per each disk seeks).

The "biggest" mirrors I manage are stored on a two-disks RAID1 volume 
(to avoid stripping which induce disks seeks) and IO are optimized by 
doing 1 MB chunk readahead (using posix_fadvise). If I need additionnal 
bandwidth, I may increase the readahead chunk size. But this is usable 
only if the file I read is big enough (and if the fragmentation is kept 
low).

As long as jigdo use is uncommon, I just don't mind but if it had to be 
commonly used, it would mean a very sensible decrease in performances.

François