[Pulp-list] grinder yum filters

[John Morris]

Hi list, I just discovered pulp the other day while looking for a more 
convenient way to synch yum repos than my crazy homebrew bash scripts 
with 'plugins' and per-repo config files.

Mainly, rsyncing whole repos is fine for the small sites I manage. 
However, a number of custom RPMs depend on small numbers of packages 
from 3rd party repos, Pulp initially looked right to synch partial repos 
with its 'whitelist' feature, but it didn't take long to trace the lack 
of this feature for remote repos down to its absence in the grinder package.

For the small shop use case, grinder is fine, so I added a quick 
filtering function to it.  The filters have their own class similar to 
pulp's, so maybe it'll be easy for someone to plumb it into pulp down 
the road.  However, the attached patch can probably be called hackish, 
so maybe by the time pulp has filtering for remote repos, it'll be a 
complete rewrite.  I don't feel too embarrassed about the hackishness, 
though, since grinder appears to be a bit of a patchwork itself; I'm 
guessing that the RHN and yum parts were two separate scripts that were 
quickly thrown together into a single library?

The patch is pretty well self-documenting.  Testing was done with 
something like the following command line:

PYTHONPATH=src python bin/grinder yum --label somerepo \
	-b /tmp -U 'http://www.somesite.com/somerepo' \
	--filter=whitelist --filter_regex='cool-package.*' --debug

BTW, 'run-tests.py' will go through its routine by adding the line 
'sys.path.append("itests/")', but lots of failures, so I didn't bother 
with unit tests.

This patch doesn't deal with filtering __getDRPMs.  It would be easy to 
add, but my sites don't need it just yet, and this submission is partly 
just to test the warmth of the waters.

This patch doesn't deal with running createrepo.  Pulp has its own 
utilities to take care of that.  With this patch, you'll end up with the 
original repodata but of course missing any packages excluded by the 
filters, so repodata must be regenerated elsehow.

More disclaimers:  I'm not a programmer.  I'm not a python programmer. 
It's the first time I've touched python in three years (last shop ran 
Perl, whew it's nice to be doing python again!).  I'm too lazy to read 
and understand the whole grinder code, so this patch probably makes no 
sense in its greater design.  I ramble too much.

Future projects:  maybe add /etc/grinder/yum.yml config (already have a 
good start), and maybe add createrepo functionality.

	John
-------------- next part --------------
A non-text attachment was scrubbed...
Name: grinder-filters.patch
Type: text/x-patch
Size: 15363 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20120322/462cfca8/attachment.bin>