mirrorlist

[Matt Domsch]

On Sun, Mar 25, 2007 at 05:54:15PM -0500, Mike McGrath wrote:
> Kostas Georgiou wrote:
> >Hi,
> >
> >I just noticed that the mirrorlist script returns the full mirror list
> >if the mirrors for your country are below a certain limit. For example
> >here is what happens with most european countries:
[snip]
> >
> >As it is now only US,CA,FR,DE get a "local" mirror as far as I can see
> >
> >I was wondering if it makes more sense to use mirrors from "nearby" 
> >countries
> >instead of the full list in cases like this. Defining "nearby" could be
> >a problem but even the same continent is bound to be better than the 
> >existing
> >system.
> >
> >Any thoughts?
> >  
> This is intentional.  The issue is a low number of mirrors for some 
> countries.  If the returned list is less then a certain number (I think 
> its 4?) Then it will return the global list.  One thing I've meaning to 
> do is re-evaluate how we determine if a mirror is out of date.  We also 
> just need more mirrors IMO.  Matt Domsch has been working on a new 
> mirror management system which should help aid this process.
>
> Matt, just curious, does your mirror app check for freshness?  Will it 
> completely replace our check-mirrors script?

I've got a crawler that runs every 6 hours comparing against the
filelist available from the master rsync server at the time the run
starts.  It's pretty fast, multi-process, using keepalive HTTP HEAD
and FTP DIR calls to determine presence and size of files - close
enough for our purposes.  So yes, it replaces check-mirrors.  It
doesn't (yet) replace mirrorlist.cgi - the app that returns the mirror
lists to the yum client.  It generates files usable by mirrorlist.cgi
as-is.  Or almost does.  That piece needs some work... :-)

I've started looking at the GeoIP library too.  Turns out, you can get
the continent from the country 2-letter ISO code, so we could return
mirrors on the same continent.  MirrorManager has a list "Countries
Allowed" on a per-Host basis, so for instance the mirror in Belgium
could dictate it can only accept clients in Belgium.  (yes, that's
actually a restriction that mirror has, but it's just described in the
html for the entry, not enforced by Fedora code anywhere yet).

As for more mirrors, we've added several in the last few months with
good connectivity in countries we haven't had previously: China,
Mexico, better bandwidth in Canada, ...  but most mirrors are still
having trouble syncing.  I don't want to add a bunch of new mirrors
until we get some of the other infrastructure pieces, namely storage
and storage bandwidth, straightened out.

Thanks,
Matt

-- 
Matt Domsch
Software Architect
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com