[et-mgmt-tools] Thoughts on repo mirroring...

[Michael DeHaan]

Michael DeHaan wrote:

First off, if this sounds like greek, let me know, and I'll try to 
explain something differently.   I'm as guilty as speaking tech lingo as 
anyone.

This is probably starting to sound like "cobbler-dev" lately and not a 
user list, though repo mirroring is starting to evolve some and some 
folks may have thoughts on it -- so I wanted to bring these items up 
rather than just secretly implement them and have to explain them later. 

Imagine you can do this:

# mirror a kickstart tree (this will take a while, but you only have to 
do it once per distro)
cobbler import --mirror-name="ibiblio_fedora6_i386" 
--mirror="rsync://distro.ibiblio.org/fedora-linux-core/6/i386"
# also add some additional repos to mirror
cobbler repo add --name="fc6i386updates" 
--mirror="rsync://distro.ibiblio.org/fedora-linux-core/updates/6/i386" 
--local-filename="fedora-extras"
# create a profile that says "use this distro and these repos"
cobbler profile add --name="fc6i386" 
--distro="var_www_cobbler_ks_mirror_ibiblio_i386_os_images_pxeboot" 
--repos="fc6i386updates"
# tell systems how to boot by default
cobbler system add --name=default --profile="fc6i386"
# update/copy the repos via rsync (this will take a while the first 
time, and then will be pretty fast)
cobbler reposync   # you'll probably want to add this to cron
# apply the cobbler configuration
cobbler sync

Ok, so in upstream mercurial code (though it needs a good deal of 
testing) -- you can do exactly that.

What do these commands do?  They create a local mirror of Fedora Core 6 
and Extras, from a public rsync mirror, that you can provision off of.  
That was there previously -- it's not news.  In addition, you've just 
configured an FC6 update mirror (that you can update at will) that any 
boxes you provision off of that profile will use for updates, by default 
-- this IS new.   So, if you get a 1000 baremetal machines in on a truck 
(Merry Christmas?) and want to set up a boot server for all of them -- 
and never have them ask the outside world for package updates, that's 
one way you could do it.   This is a good way to set up boxes that know 
about custom repositories as well.  Before cobbler mirrored the 
kickstart tree, which was nice, but there was no concept of mirroring 
packages not in that original tree -- which is very closely related and 
just as important when we are talking about "mirroring" as opposed to 
just finding some way to make installs happen.

This _will_ probably change some, but I wanted to pass along the 
direction of where I wanted to take the boot server.  

I may be looking at Pungi (http://jkeating.livejournal.com/32250.html) 
some also, though support for non-Pungi compatible distros needs to be 
maintained in some form.   Pungi would be interesting in that we could 
define a cobbler distribution as a set of repository URLs and packages, 
rather than defining it as a kernel, initrd, kickstart tree, and other 
repos (basically it would be a lot simpler).   Basically it's a custom 
repo and install CD composing tool.   Again though, non-Pungi compatible 
distros need to have some support too, and that _might_ not extend far 
beyond the rsync:// support that is there now. 

Fire away if you have any comments/ideas/questions on this...  "I'm 
confused" is ok too :)

--Michael