[Pulp-list] Handling Uploads to repos with feed

[Mike McCune]

On 10/11/2010 12:49 PM, tsanders at redhat.com wrote:
> Can someone help me understand a use case where you would want to combine the
> contents of 2+ external yum repos into a single pulp repo?  Flexibility is
> great, but we need to be sure that this functionality is worthwhile.
>

I was less concerned with combining 2 external yum repos but more with 
the case where I have content coming in from an external yum repo and a 
series of uploads using the upload CLI/API.

I agree the mashing of 2 externally separate yum repos into one bigger 
repo is a bit less useful than just adding extra packages through uploads.

> I get being able to mirror fedora, the Dell firmware repo, and setting up a
> custom repo for my internally developed tools.  Not sure I would ever want all
> of this in a single repo, that would seem to be confusing.  Is having my
> systems subscribed to one repo that much more beneficial than subscribed to 3?


It seems confusing because we are so accustomed to having everything 
separated out by channels in the 'channels gone wild' world of 
Satellite.  but in order to clarify where I am coming from here is a 
specific ...

== Use Case ==

* User has a number of systems using pulp to get their content.  They 
have a few versions of Fedora synced into various repos by version and arch.

* User thinks to themselves:

"I have my Fedora 13 systems that I also want to just get these few 
extra packages from my internal dev group available to my systems.. let 
me just upload them to the repo so they are there"

$ pulp-admin repo upload --id=f13-x86_64 --dir=/mnt/extra-packages

ERROR: You can't upload to repo [f13-x86_64] because it has a feed defined.
You can only upload packages to feedless repos

Now the user has to create a separate repo, upload his packages and then 
get all his systems bound to that new repo.

This is because we decided to block uploads to repos with feeds defined.

Now ... this aside, I do see were you all are coming from that this use 
case may be a bit less common and more of a corner-case but I still 
don't like the 'baking-in' of an actual rule into pulp that doesn't 
allow users to be able accomplish the above.

If implementing the above flexibility causes massive headaches and it 
turns out we should spend our time working on other things that are more 
important to our users then I am definitely fine with that.

== Proposal ==

Could we instead at least do what I had suggested earlier about an 
'allows_uploads' flag?  if this flag is set then syncs will be a bit 
slower because we have to run createrepo --update at the end of the sync 
to combine the repodata synced as well as the packages also in the repo 
from possible uploads.

If it is not set (and I'd even propose we have it set to false by 
default) we rely on the external metadata and instead the error message 
above would be:

$ pulp-admin repo upload --id=f13-x86_64 --dir=/mnt/extra-packages

ERROR: You can't upload to repo [f13-x86_64] because it has 
'allow_uploads' set to false.  To enable uploads to [f13-x86_64] just 
"pulp-admin repo edit --allow-uploads=true --id=f13-x86_64"

Then in the future if we want to expand to add more policy around what 
can and can't be done to repos we are teaching the users there are rules 
around what can be done vs just saying 'no'.

If I'm over simplifying what needs to be done to support the above 
*please* feel free to correct me.

Mike