[Pulp-dev] Name uniqueness problem in Pulp 3 REST API

Wed Jul 22 08:55:24 UTC 2020

Sorry for my short interjection. I was in a hurry and needed to make
sure i won't need to "hold my peace once and for all".
I am still quite worried, when you say "to change the uniqueness
constraint for repositories", as it is not clear to me whether it
means to lift the constraint completely or to change the set of
db-fields that represent the natural key of the resource.
Let me try to elaborate what i mean by the "ansible use case":
Consider a playbook to create a repository, a remote and trigger a
sync. It might look roughly like:
---
- file_repository:
    name: my_namespace/my_repository
    state: present
- file_remote:
    name: my_namespace/my_remote
    url: http://example.org
    state: present
- file_sync:
    repository: my_namespace/my_repository
    remote: my_namespace/my_remote
...
Note, that there are no uuids in the playbook, because you cannot know
them upfront. Also if you run the playbook a second time, referencing
the resources by their natural key (that is the name atm) allows the
playbook to be idempotent. There will not be a second repository
created. That said, i am ok with changing the "natural
key"/"uniquene_together" to ["name", "namespace", "type"], but do not
want to see it dropped.
Additionally to that, i think it makes for a bad user experience too,
if a user needs to remember uuids of repositories, because there is
nothing else to identify them.
  Matthias

On Tue, Jul 21, 2020 at 11:08 PM Brian Bouterse <bmbouter at redhat.com> wrote:
>
> Grand and David and I met for a *long* conversation about all the issues here. I'll try to summarize some of the highlights. Nothing was decided that was clear.
>
> * Grant's language about multi-tenancy is helpful, that is at the heart of my concern
> * I believe we need multi-tenancy for Pulp to be safe and achieve the project's goals, I've added a multi-tenancy discussion to the pulpcon agenda if we don't get to it sooner https://hackmd.io/hIOjFsFiSkGJR7VqtAJ8eQ
> * Generally namespaces or some other form of multi-tenancy mechanisms seems to be the solution
> * I agree it won't resolve situations where there are data conflicts around URLs, e.g. with Distributions specifically
>
> So for this thread, if it's helpful to change the uniqueness constraint for repositories that seemed fine by everyone I've talked to. I'm interested in organizing an effort to make sure we don't introduce RBAC that users dislike with 3.6 but that needs more comprehensive planning. If anyone has a simple proposal on how to do that please let me know. I'll likely draft a problem statement here soon and probably collab w/ @ggainey on it.
>
> On Tue, Jul 21, 2020 at 12:38 PM Matthias Dellweg <mdellweg at redhat.com> wrote:
>>
>> The user story you are describing there is inevitably not satisfiable.
>> At the very least, not every user can randomly create distributions
>> with base paths colliding with existing distributions.
>> I believe the answer to that is namespaces, where users can create
>> entities in their namespaces that they can also see. BTW, the
>> namespace could be used to decide upon a certain group permission to
>> be added to new resources.
>> In the global namespace, only an administrator can create entities.
>> If you lift, the uniqueness of names in a certain content-type, you
>> are definitely breaking the ansible usecase.
>>
>> On Tue, Jul 21, 2020 at 5:56 PM Grant Gainey <ggainey at redhat.com> wrote:
>> >
>> > On Tue, Jul 21, 2020 at 11:38 AM Dennis Kliban <dkliban at redhat.com> wrote:
>> >>
>> >> On Tue, Jul 21, 2020 at 11:22 AM Brian Bouterse <bmbouter at redhat.com> wrote:
>> >>>
>> >>> I'm concerned if we don't make a change, here's the user experience I'm worried about.
>> >>>
>> >>> 1. User A creates repo 'rhel7'
>> >>> 2. user B can't see repo 'rhel7' because of queryset scoping
>> >>> 3. user B goes to create 'rhel7'
>> >>> 4. user B is told 'rhel7' already exists
>> >>>
>> >>> Users should be able to use simple names. I don't know what the answer is to the import/export implementation conflict, but let's brainstorm some. For the benefit of our users, I don't think that implementation should interfere with this basic use.
>> >>>
>> >>
>> >> I agree that this is a usability problem for our users.
>> >>
>> >> With regard to import/export, the ideal solution would use the same UUID in both the system that's exporting and the system that's importing. Is my understanding correct?
>> >
>> >
>> > For PIE to work, it needs to be able to identify whether something needs to be created otr updated in the 'downstream', and therefore needs to be able to identify instances as being "the same thing". pulp_id definitely does that. However, the use-case for PIE can't rely on pulp_id, because it's not a replica of upstream. Consider the migrated-use-case - starting from pulp2, I have the exact same content in upstream and downstream, but completely different pulp_ids.
>> >
>> > In any event - PIE isn't really the major issue as far as I am concerned, it's deciding if a single pulp instance is multi-user or multi-tenant and following the implications from there.
>> >
>> > G
>> >
>> >>
>> >>
>> >>
>> >>>
>> >>> Side note: from early on in Pulp3, pk's not names have been the primary identifier. I'm unclear on how we got away from that.
>> >>>
>> >>>
>> >>>
>> >>> On Tue, Jul 21, 2020 at 9:03 AM Matthias Dellweg <mdellweg at redhat.com> wrote:
>> >>>>
>> >>>> I always understood the "lifting the uniqueness" as allowing to have
>> >>>> the same name used for different resource types. So the new
>> >>>> natrual_key (aka unique_together) would be ["name", "type"].
>> >>>>
>> >>>> On Tue, Jul 21, 2020 at 2:55 PM David Davis <daviddavis at redhat.com> wrote:
>> >>>> >
>> >>>> > Agreed.
>> >>>> >
>> >>>> > David
>> >>>> >
>> >>>> >
>> >>>> > On Tue, Jul 21, 2020 at 8:42 AM Grant Gainey <ggainey at redhat.com> wrote:
>> >>>> >>
>> >>>> >> On Tue, Jul 21, 2020 at 8:14 AM Dennis Kliban <dkliban at redhat.com> wrote:
>> >>>> >>>
>> >>>> >>> Does anyone else have an opinion? If not, I am going to start by writing a task to remove this name uniqueness constraint for repositories.
>> >>>> >>
>> >>>> >>
>> >>>> >> Import/export relies on non-pulp_id-uniqueness to identify Things. I was assuming we were talking about adding pulp_type to the Repository uniqueness-constraint, so that a given name/type would be unique (which would require a single change to RepositoryResource)
>> >>>> >>
>> >>>> >> If we're talking about just removing the uniqueness-constraint altogether, then life gets a lot harder.
>> >>>> >>
>> >>>> >> G
>> >>>> >> --
>> >>>> >> Grant Gainey
>> >>>> >> Principal Software Engineer, Red Hat System Management Engineering
>> >>>> >> _______________________________________________
>> >>>> >> Pulp-dev mailing list
>> >>>> >> Pulp-dev at redhat.com
>> >>>> >> https://www.redhat.com/mailman/listinfo/pulp-dev
>> >>>> >
>> >>>> > _______________________________________________
>> >>>> > Pulp-dev mailing list
>> >>>> > Pulp-dev at redhat.com
>> >>>> > https://www.redhat.com/mailman/listinfo/pulp-dev
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Pulp-dev mailing list
>> >>>> Pulp-dev at redhat.com
>> >>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>> >>>>
>> >>> _______________________________________________
>> >>> Pulp-dev mailing list
>> >>> Pulp-dev at redhat.com
>> >>> https://www.redhat.com/mailman/listinfo/pulp-dev
>> >>
>> >> _______________________________________________
>> >> Pulp-dev mailing list
>> >> Pulp-dev at redhat.com
>> >> https://www.redhat.com/mailman/listinfo/pulp-dev
>> >
>> >
>> >
>> > --
>> > Grant Gainey
>> > Principal Software Engineer, Red Hat System Management Engineering
>> > _______________________________________________
>> > Pulp-dev mailing list
>> > Pulp-dev at redhat.com
>> > https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>