InstantMirror Proposal Re: ApacheMirror.py for a site-local Fedora mirror

[Dmitry Butskoy]

Warren Togami wrote:

> Perhaps a separate, asynchronous daemon can monitor upstream (via HTTP 
> or whatever) for repomd.xml changes.  It should then parse the 
> repomd.xml so it knows when to expire the repodata/* files.  Then it 
> should parse the .xml files in repodata/ to compare it to local 
> storage, and intelligently expire the packages if any changed (as 
> happens during signing).  It can then know exactly which files to 
> delete from the local cache because they are no longer in the upstream.  

I have thought about something similar already.

Perhaps I'll speak about something different. :)

I've created a simple yum plugin, "yum-justdb". With an alternate "root" 
(--installroot=DIR) and "keepcache=1", it allows to "install", "update", 
"remove" etc. packages "virtually" -- i.e. against a special separate 
rpm database. By this way, all transactions are performed (all 
dependency checking, obsoleting etc.), and the downloaded packages can 
be simple obtained by the yum cache (then moved to a local repository by 
shell scripts etc.)

I've written some shell script which utilize all the tasks needed to 
maintain a local "partial repo" (a subset of the original 
Fedora+Livna+others repos, with all dependencies resolved). Now I'm 
testing it.

The "rpm" command always has "--root DIR" and "--justdb" options, but 
YUM currently has "--installroot=DIR" only, hence the additional 
yum-justdb plugin is needed. But by design, YUM already has an option 
"tsflags" (and the correspond yum-tsflags plugin in yum-utils). 
Unfortunately, "tsflags" supports only subset of possible rpm 
transaction flags, and "justdb" is not supported yet.

I've already proposed the simple patch for "justdb": 
https://lists.dulug.duke.edu/pipermail/yum/2007-November/010310.html
but it seems nobody hear me.

I do not want to publish my script until I will be clear whether I 
should publish an extra yum plugin as well, or just refer to the latest 
yum release with "tsflags=justdb" support. Just because the extra plugin 
will be unneeded then, and I have to retrain users from "yum --justdb" 
to "yum --tsflags=justdb" .

If anyone can promote the adding of 'justdb' support to yum, it could be 
nice. Then we can start to work further.


Regards,
Dmitry Butskoy
http://www.fedoraproject.org/wiki/DmitryButskoy