[et-mgmt-tools] Thoughts on repo mirroring...

[Michael DeHaan]

Formatting fixed (somewhat) below...

Michael DeHaan wrote:
> Added some preliminary support for making local mirrors of yum 
> repositories (rsync mirrors only thus far).   Nothing incredibly 
> magical, but it could prove useful to some as it evolves.
> It's checked in, but not documented (except here).
>
> Example:
>
> # cobbler repo add --name=fc6i386updates 
> --mirror=rsync://distro.ibiblio.org/fedora-linux-core/updates/6/i386

> # cobbler reposync 

> # cobbler profile add --name=blah --distro=blah 
> --kickstart=/tmp/foo.ks --repos="fc6i386updates"
>
> The "reposync" command would rsync any repo's cobbler knew about, and 
> is called independently from sync.   It's suitable for usage on cron, 
> though cobbler doesn't install a crontab yet.   Later, it will 
> probably be expanded
> to support more types of mirroring than just rsync (i.e. reposync or 
> using the yum API).

> Now, the first two commands just provide basic mirroring.   The last 
> command includes the "--repos" bit, which is only useful on FC6 and 
> later (not sure about RHEL5?).   If the foo.ks assigned to the profile 
> contains the template line "TEMPLATE::yum_repo_stanza", that spot in 
> the template would be replaced with the code to let Anaconda install 
> from any repos assigned to that profile.   The value of the replaced 
> "TEMPLATE::yum_repo_stanza" entry would be ...  

>    repo --name=fc6updates 
> --baseurl=http://172.16.57.21/repo_mirror/fc6updatesi386
>
> The "/etc/cobbler/kickstart_fc6.ks" that ships with cobbler (from now 
> on) has that TEMPLATE::yum_repo_stanza line in there to be used if any 
> repos are configured.
> You can also specify more than one repo, i.e. "cobbler profile add ... 
> --repos="fc6i386updates fc6i386extras mycompanyrepo" and so forth.  . 

> This is just a start, but it's a logical progression of mirroring, 
> since the point of having a local boot server is partially to have 
> speedy OS installs -- that should apply to upgrades too.
>
> Note that this doesn't configure yum.repos.d on the installed machine 
> -- you'll still have to do that on your own (for now).    Cobbler 
> really shouldn't force puppet or cfengine at anyone who doesn't want 
> it (yet) and those that
> are already using one can use those tools to push out the mirror 
> configurations.
>
> Michael DeHaan wrote:
>
>>
>> A few thoughts on mirroring...
>>
>> "Cobbler import" provides a way of keeping the network demands of 
>> running a large install farm under control -- that is, kickstart tree 
>> content doesn't have to come from public mirrors.   This means faster 
>> installs, less bandwidth, and in general being nice to community 
>> provided mirrors.   One thing that cobbler hasn't been doing is to 
>> integrate this with a good repository mirroring strategy.    Installs 
>> from public and private data ought to come in through the boot 
>> server, not the local university every time.
>>
>> I'm looking fairly closely at reposync and plain rsync right now... 
>> rsync would have some disadvantage in that it would generally bring 
>> down too much content, for instance, say you weren't interested in 
>> KDE.   Comments?   Thoughts?   What tools are folks already using to 
>> do this?
>>
>> Tenatively, I see this looking something like
>>
>> cobbler repo add --name=foo --mirror=URL
>> and putting something like
>> "cobbler repo sync" on a crontab.
>>
>> Then, each cobbler profile would have a new optional property 
>> ("--repos") that would give information about what repositories to 
>> use by default.
>>
>> cobbler profile add --name=p1 --distro=d1 --repos="fc6updates 
>> fc6extras myspecialfc6repo"
>>
>> If a distribution being detected in an import is greater than "FC6", 
>> we know that we can add repo's directly in the kickstart based on the 
>> "repo" directive, otherwise, we'll have to tweak the kickstarts 
>> slightly differently.     All of the repo assignment could be 
>> automagically filled in during kickstart templating (which is done at 
>> "cobbler sync" time).
>>
>> --Michael
>>
>> _______________________________________________
>> et-mgmt-tools mailing list
>> et-mgmt-tools at redhat.com
>> https://www.redhat.com/mailman/listinfo/et-mgmt-tools
>