[RFC]InstantMirror Redesign

[Bill Crawford]

On 10/03/2008, Kulbir Saini <kulbirsaini at students.iiit.ac.in> wrote:

>     1. InstantMirror to be used by a small group of people. In this case we
>  have to get rid of all the dependencies like squid, apache because for a
>  small setup nobody is going to configure squid and apache. So, we use a
>  proxy server implemented in python for this kind of setup and integrate
>  InstantMirror with it in caching mode. So that it becomes easy to setup
>  and don't require squid, apache or whatever else.

You've just described "squid in offline mode", more or less, above
(offline being the "honour request from cache if we're unable to
connect" part).

>     2. InstantMirror to be used by an organization. As almost all the
>  organization ( i am focusing more on institutes/universities here) use a
>  common proxy server to access the Internet, we will have the InstantMirror
>  which can be integrated with squid. There will be no difficulty in setup
>  as the people already use squid (assuming squid is widely used in
>  Unix/Linux world) and know how to configure it. We can't use proxy server
>  implemented in python here because no organization would ever agree to use
>  a stripped down version of proxy server instead of squid.

... and in this case, you want squid. Basically.

>     Imagine a university with thousands of Linux users and everyone is
>  updating their system weekly. GBs of bandwidth is being wasted every week
>  due to subsequent downloads of the same package.

You just have to update the "maximum object size" for the squid cache
to cover the largest package in the distro. It's more or less plug and
play after that. You will need to make all your clients use the same
mirrorlist for all your clients, of course :))

>     If you have any suggestions for improvements, comments on the current
>  design or you want to criticize the design, please reply back. They would
>  really help me to improve.

Seriously, you will end up rewriting squid. Just use it :o) it's
surprisingly light on memory if you configure it right. You wouldn't
mind a single-purpose proxy running on your machine? You would have
the same benefit from actually using squid, *plus* that can be used
for other purposes (i.e. shared cache for browsers running on the
machine, too). I've done this for many years now.

>  PS : This is my first RFC, if i wrote it badly please forgive me :)

It's not bad at all! You've covered all the bases I could see. You
would still need a "mirror manager" process involved (to do any kind
of prefetching). even with squid as a proxy; so please don't take the
above as any kind of slap in the face. You've done a good job of the
requirements doc :)