[Pulp-list] Handling Uploads to repos with feed

[Todd B. Sanders]

  On 10/11/2010 05:59 PM, Mike McCune wrote:
> On 10/11/2010 12:49 PM, tsanders at redhat.com wrote:
>> Can someone help me understand a use case where you would want to 
>> combine the
>> contents of 2+ external yum repos into a single pulp repo?  
>> Flexibility is
>> great, but we need to be sure that this functionality is worthwhile.
>>
>
> I was less concerned with combining 2 external yum repos but more with 
> the case where I have content coming in from an external yum repo and 
> a series of uploads using the upload CLI/API.
>
> I agree the mashing of 2 externally separate yum repos into one bigger 
> repo is a bit less useful than just adding extra packages through 
> uploads.
>
>> I get being able to mirror fedora, the Dell firmware repo, and 
>> setting up a
>> custom repo for my internally developed tools.  Not sure I would ever 
>> want all
>> of this in a single repo, that would seem to be confusing.  Is having my
>> systems subscribed to one repo that much more beneficial than 
>> subscribed to 3?
>
>
> It seems confusing because we are so accustomed to having everything 
> separated out by channels in the 'channels gone wild' world of Satellite.
Agreed, but with the combination of environments and cloning we should 
be able to limit the issues around proliferation of repos in Kalpana.
> but in order to clarify where I am coming from here is a specific ...
>
> == Use Case ==
>
> * User has a number of systems using pulp to get their content.  They 
> have a few versions of Fedora synced into various repos by version and 
> arch.
>
> * User thinks to themselves:
>
> "I have my Fedora 13 systems that I also want to just get these few 
> extra packages from my internal dev group available to my systems.. 
> let me just upload them to the repo so they are there"
>
> $ pulp-admin repo upload --id=f13-x86_64 --dir=/mnt/extra-packages
>
> ERROR: You can't upload to repo [f13-x86_64] because it has a feed 
> defined.
> You can only upload packages to feedless repos
>
> Now the user has to create a separate repo, upload his packages and 
> then get all his systems bound to that new repo.
>
> This is because we decided to block uploads to repos with feeds defined.
>
> Now ... this aside, I do see were you all are coming from that this 
> use case may be a bit less common and more of a corner-case but I 
> still don't like the 'baking-in' of an actual rule into pulp that 
> doesn't allow users to be able accomplish the above.
>
> If implementing the above flexibility causes massive headaches and it 
> turns out we should spend our time working on other things that are 
> more important to our users then I am definitely fine with that.

Definitely see your point, however, I would argue the alternative 
viewpoint.... that creating a separate repo within Pulp is super simple, 
along with binding the newly added repo to my consumer group.  Further 
more, I see benefit in keeping the custom package separate from the core 
Fedora repos from a mgmt perspective.  Lastly, using repo groups, you 
could additionally group the repos together for simplicity of mgmt.

>
> == Proposal ==
>
> Could we instead at least do what I had suggested earlier about an 
> 'allows_uploads' flag?  if this flag is set then syncs will be a bit 
> slower because we have to run createrepo --update at the end of the 
> sync to combine the repodata synced as well as the packages also in 
> the repo from possible uploads.

Kalpana case: How can we prevent customers from just changing this flag 
for RH provided products/repos?

I'd imagine it would be nice from a RH support perspective to know that 
the integrity of the RH provided products/repos are kept in tact (no 
additions or removals); especially since we will require entitlement 
consumption to access....not only for orgin repo but any clones as well.

-Todd
>
> If it is not set (and I'd even propose we have it set to false by 
> default) we rely on the external metadata and instead the error 
> message above would be:
>
> $ pulp-admin repo upload --id=f13-x86_64 --dir=/mnt/extra-packages
>
> ERROR: You can't upload to repo [f13-x86_64] because it has 
> 'allow_uploads' set to false.  To enable uploads to [f13-x86_64] just 
> "pulp-admin repo edit --allow-uploads=true --id=f13-x86_64"
>
> Then in the future if we want to expand to add more policy around what 
> can and can't be done to repos we are teaching the users there are 
> rules around what can be done vs just saying 'no'.
>
> If I'm over simplifying what needs to be done to support the above 
> *please* feel free to correct me.
>
> Mike