[Pulp-dev] uniqueness constraints within a repository version

Tue Jun 25 18:02:59 UTC 2019

I think I misread your email. If you are saying "newest to associate" and
not "newest content unit", I think that would work.

@ttereshc, couldn't we de-duplicate the logic by creating a class in the
plugin API that RemoveDuplicates uses as well as the add/remove content
endpoints in the plugins?

David

On Tue, Jun 25, 2019 at 12:54 PM David Davis <daviddavis at redhat.com> wrote:

> I don't think this solution would work in the case of creating a new
> repository version. Suppose for example you had two content units that
> collide, one in a repo version and one older unit that a user explicitly
> wants to add to the repo version. If the latter one is older, then what
> would happen?
>
> David
>
>
> On Tue, Jun 25, 2019 at 12:48 PM Brian Bouterse <bbouters at redhat.com>
> wrote:
>
>> Having a way for units to express their uniqueness per repo sounds good
>> because then more areas of Pulp's code could answer the question: "will I
>> have a duplicate if I add content X to repo_version Y".
>>
>> Let's assume we know that situation is about to occur during sync for
>> example, what do we do about it? In the errata case we know the "new" one
>> should replace the existing one. Maybe we start to 'order' the units with
>> colliding repo keys and keep the newest one always? Would this work for
>> pulp_cookbook and pulp_rpm? Would it generalize? Is this what you imagined?
>>
>> On Tue, Jun 25, 2019 at 5:30 AM Tatiana Tereshchenko <ttereshc at redhat.com>
>> wrote:
>>
>>> Do I understand correctly that it doesn't cover the sync case and it's
>>> only about explicit repo version creation?
>>> So the suggestion is to implement the same logic twice: for sync case -
>>> RemoveDuplicates stage and/or maybe some custom stage (e.g. to disallow
>>> overlapping paths), and for direct repo version creation - your proposal.
>>>
>>>
>>>
>>> On Mon, Jun 24, 2019 at 3:13 PM Austin Macdonald <amacdona at redhat.com>
>>> wrote:
>>>
>>>> I have a design in mind for solving this problem:
>>>>
>>>> 1. Remove POST to RepositoryVersion (no general add/remove endpoint).
>>>> 2. Add an endpoint to kick off an add/remove task, namespaced by
>>>> plugin. ie `POST pulp/api/v3/docker/add-remove/`
>>>>    This view can be provided to all plugins by the plugin template, and
>>>> will be based on the current RepositoryVersionCreate:
>>>>
>>>> https://github.com/pulp/pulpcore/blob/master/pulpcore/app/viewsets/repository.py#L221-L258
>>>>    Note: the main purpose of this view is to kick off the general
>>>> add/remove task, which will be unchanged:
>>>>
>>>> https://github.com/pulp/pulpcore/blob/master/pulpcore/app/tasks/repository.py#L70
>>>> 3. Add an add/remove serializer to the plugin API.
>>>> 3. Plugins needing further customization can provide their own task and
>>>> subclassed serializer.
>>>>
>>>> This gives the plugin writer full control over the endpoint
>>>> (customizable arguments and validation), and full control over the flow
>>>> (extra logic, depsolving, enforced uniqueness). It only uses the existing
>>>> patterns (and existing required knowledge), but requires no work (other
>>>> than using the template) for the simple case.
>>>>
>>>> On Mon, Jun 3, 2019 at 2:56 PM Simon Baatz <gmbnomis at gmail.com> wrote:
>>>>
>>>>> On Mon, Jun 03, 2019 at 09:11:07AM -0400, David Davis wrote:
>>>>> >    @Simon I like the idea behind the repo_key solution you came up
>>>>> with.
>>>>> >    Can you be more specific around cases you think that it couldn't
>>>>> >    handle? I imagine that plugin writers could use properties or
>>>>> >    denormailzation (ie additional database columns) to solve cases
>>>>> where
>>>>> >    they need uniqueness across data that isn't in the database. In a
>>>>> worst
>>>>> >    case scenario, they can't use the pulpcore solution and just have
>>>>> to
>>>>> >    roll their own.
>>>>>
>>>>>
>>>>> What I wrote probably sounded too pessimistic. You are right, in
>>>>> most cases that should be doable.
>>>>>
>>>>> I agree that we could have a simple default solution that just
>>>>> requires to specify a couple of field names in the easiest case.  As
>>>>> you
>>>>> say, it should be possible use custom logic in a plugin if required.
>>>>>
>>>>> Here is the case I was thinking of that it can't handle:
>>>>>
>>>>> In pulp_file, a uniqueness constraint on "relative_path" would allow
>>>>> content units "a" and "a/b" to be in a repo version.
>>>>>
>>>>> However, we may want file repos to be representable on an actual file
>>>>> system (e.g. when exporting them as tar files).  For the repo above,
>>>>> this does not work, as "a" can't be a file and a directory at the
>>>>> same time on a standard Unix file system.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190625/f31c140a/attachment.htm>