<div dir="ltr"><div>Thanks to @asmacdo for writing a ticket description of the work here [0]. It includes a description of what the API would look like.</div><div> </div><div>@mhrivnak I agree, I also want to see the url that lists all publishers for a given repo. I think what you've written or similar is what I expect. </div><div> </div>I've seen a lot of +1s. Are there any -1's to it being groomed and proceeding? Do we want more time with this decision? It's hard to know on mailing lists when we've reached that point. I bring it up just because it's blocking several areas of work. <div> </div><div>Thanks to everyone for all the conversation!</div><div> </div><div>[0]: <a href="https://pulp.plan.io/issues/3160">https://pulp.plan.io/issues/3160</a></div><div> </div><div>-Brian</div><div> </div></div><div class="gmail_extra"> <div class="gmail_quote">On Thu, Nov 30, 2017 at 4:06 PM, Jeff Ortel <<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On 11/29/2017 04:32 PM, Brian Bouterse wrote: > For deletes, the db relationships are all there, so I expect deletes to cascade to other objects with any url > structure. I believe closer to the release, we'll have to look at the cascading delete relationships to see if > the behaviors that we have are correct. > > Overall, I'm +1 on un-nesting. I think it would result in a good user experience. I know it goes against the > logical composition arguments, which have been well laid out. We want Pulp to be really simple, and the nested > URL in the top of this thread is anything but simple. Consider another project like Ansible Galaxy (who also > uses Django and DRF). Their API is very flat and as an outsider I find it very approachable: > <a href="https://galaxy.ansible.com/api/v1/" rel="noreferrer" target="_blank">https://galaxy.ansible.com/api/v1/</a> Pulp could be that simple. Clicking through the Galaxy API, there seems to be a good bit of nesting. > > My main concern in keeping the nesting is that this is going to be difficult for plugin writers. Making plugin > writing easy is a primary goal if not the primary goal of Pulp3. If core devs are spending lots of time on it, > a person doing this in their free time may not bother. > > I also see practical reasons motivating us to un-nest. We have been adding custom code regularly in this area, > and it's been highly complexity and slow going. I think Austin described it well. Getting the viewsets working > and to be simpler would allow us to move forward in many areas. > > So overall, un-nesting would give a better user experience (I think), a simpler plugin writer experience, and > it would unblock a lot of work. > > > > On Wed, Nov 29, 2017 at 3:29 PM, Bihan Zhang <<a href="mailto:bizhang@redhat.com">bizhang@redhat.com</a> <mailto:<a href="mailto:bizhang@redhat.com">bizhang@redhat.com</a>>> wrote: > > I have a question about repository delete with the un-nested model. > When a repository is deleted does the DELETE cascade to the importers/publishers that are linked to the > repo? In an un-nested world I don't think they would. It would be odd for an object with its own endpoint > to vanish without the user calling DELETE on the model. > > When nested it makes sense to cascade the delete so if /repo/1/ is deleted, everything thereafter > (/repo/1/importer/2) should also be removed. > > Austin, I do see you point about it being a lot more complicated, but I think modeling things the right > way is worth carrying the extra code and complexity. > > Anyways, maybe I'm wrong and importer/publishers should exist without a repository, in which case I can > definitely see the value in un-nesting the URLs. > > <div><div class="h5">> On Wed, Nov 29, 2017 at 2:21 PM, Jeff Ortel <<a href="mailto:jortel@redhat.com">jortel@redhat.com</a> <mailto:<a href="mailto:jortel@redhat.com">jortel@redhat.com</a>>> wrote: > > Austin makes a compelling argument. > > > On 11/28/2017 02:16 PM, Austin Macdonald wrote: > > When I look at this, the most important point is that we have a hyperlinked REST API, which means > that the > > urls are specifically not going to be built by users. > > > > For a user to retrieve an importer, they would first GET the importers for a repository. The next > call would > > be the exact href returned by pulp. This workflow is exactly the same whether we nest or not. The only > > difference is that we no longer convey the information in the href, which seems fine to me since > they aren't > > particularly readable anyway. > > > > It has already been discussed that filtering can make up for the use cases that use nesting, and > that filters > > would be more flexible. > > > > So for me, nesting costs in (1) extra code to carry (2) extra dependency (3) complexity to use. > > > > To elaborate on the complexity, the problem is in declaring fields on the serializer. The serializer is > > responsible for building the urls, which requires all of the uuids for the entire nested structure. > This is > > further complicated by master/detail, which is an entirely Pulp concept. > > > > Because of this, anyone working on the API (likely including plugin writers) will need to understand > > parent_lookup_kwargs and how to use then with: > > DetailNestedHyperlinkedRelatedField > > DetailNestedHyperlinkedidentityField > > DetailwritableNestedUrlRelatedField > > DetailRelatedField > > DetailIdentityField > > NestedHyperlinkedRelatedField > > HyperlinkedRelatedField. > > > > The complexity seems inherrent, so I doubt we will be able to simplify this much. So, is all this > code and > > complexity worth the implied relationship in non-human-friendly urls? As someone who has spent a lot > of time > > on this code, I don't think so. > > > > > > > > On Nov 28, 2017 06:12, "Patrick Creech" <<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a> <mailto:<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a>> </div></div><div><div class="h5">> <mailto:<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a> <mailto:<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a>>>> wrote: > > > > On Mon, 2017-11-27 at 16:10 -0600, Jeff Ortel wrote: > > > On 11/27/2017 12:19 PM, Jeff Ortel wrote: > > > > > > > > > > > > On 11/17/2017 08:55 AM, Patrick Creech wrote: > > > > > One of the things I like to think about in these types of situations is, "what is good rest > > > > > api > > > > > design". Nesting resources under other resources is a necessary part of good api design, and > > > > > has > > > > > its place. To borrow some terms from domain driven development: > > > > > > > > > > Collections of objects are called aggregates. Think 'an order and its line items'. Line > > > > > items make > > > > > no sense without having the order context, so they are an aggregate that is accessed under an > > > > > Order. This is called the aggregate root. The rest api design for such an object, using > > > > > order as > > > > > the aggregate root, would look like: > > > > > > > > > > '/orders/' -- all orders > > > > > '/orders/{order_key}/' -- a specific order with key. > > > > > '/orders/{order_key}/items/' -- All of the order's items. > > > > > '/orders/{order_key}/items/{item_key}/' -- a specific line item of the order > > > > > > > > > > When it comes to order items themselves, it isn't helpful to start with them as their own > > > > > aggregate > > > > > root in one large collection: > > > > > > > > > > '/items/' -- all order items in the system > > > > > > > > The order/items is a good example of aggregation (or composition) and I agree it makes a strong > > > > case for > > > > nesting. In pulp, a repository is easily thought of as a collection or aggregation of content. > > > > > > > > > > > > > > Because you lose the order context. Based on api design, this endpoint will need to respond > > > > > with all > > > > > order items across all orders and resort to parameter filtering to provide the context you > > > > > need. > > > > > > > > > > A quote borrowed from Martin Fowler [0] > > > > > > > > > > "An aggregate will have one of its component objects be the aggregate root. Any references > > > > > from > > > > > outside the aggregate should only go to the aggregate root. The root can thus ensure the > > > > > integrity > > > > > of the aggregate as a whole." > > > > > > > > > > Publishers, importers, and publications are all aggregates that don't make much sense outside > > > > > of > > > > > their aggregate root of Repository. They are dependent on the Repository context, and from a > > > > > domain > > > > > view, should be accessed starting with their specific Repository endpoint. > > > > > > > > I don't think the aggregation relationship exists between repository and > > > > importer/publisher. There is a > > > > strong association between repository and importer/publisher which /could/ even be characterized > > > > as > > > > "ownership". However, I don't think there is an aggregation (or composition) relationship. The > > > > same for > > > > publisher & publication. A publication is associated to its creating publisher but the > > > > publisher isn't an > > > > aggregation of publications. The relationship mainly provides linkage to the repository. > > > > > > This is not an argument to flatten the URLs but meant to clarify the relationships. > > > > I'm in agreement here. I was possibly a little hasty in lumping all things that have a Repositoy fk > > as being 'dependent' in that paragraph during the formation of my argument. > > > > > > > > > > > > > > > > -------------------------------------------------------------- > > > > > Specific items rebuttals: > > > > > > > > > > Yes, using the primary key uuid's as the immutable key adds some human readable challenges > > > > > to > > > > > the API. That sounds more like a point to discuss in the human readable vs. not human > > > > > readable > > > > > immutable key debate. > > > > > > > > Agreed. > > > > > > > > Also, I don't think nesting impacts URL readability. > > > > > > > > > > > > > > One of the challenges in software engineering is ensuring the tools you are using don't > > > > > limit > > > > > your choices. DRF limited the choices for pulp's rest API design, and drf-nested-routers was > > > > > introduced to help remove that limit. If working around these limitations is complex, take > > > > > advantage of open source here and help improve the upstream dependencies for your workflow. > > > > > > > > > > As far as making things simpler for plugin writers, perhaps there are ways you can > > > > > simplify it > > > > > for them by providing some encapsulation in pulp's core instead. Abstract away the nasty bits > > > > > behind the scenes, and provide them with a simpler interface to do what they need. > > > > > > > > > > With respect to the invested time already in making this work, I agree with jeremy that it > > > > > should be considered part of the sunken cost fallacy. What does need to be evaluated though > > > > > is how > > > > > much time re-architecting at this point will cost you (discussion, planning, and development) > > > > > vs the > > > > > amount of time it will save, and weigh that against any planned milestones for pulp to see if > > > > > it > > > > > will push them out as well. > > > > > > > > > > I'm also in agreement that it is moot if pulp3 has a different api structure than > > > > > pulp2. Major > > > > > version boundaries are the perfect time for evaluating and moving such things around. > > > > > > > > > > [0] <a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a> > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a>> > > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a> > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a>>> > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Pulp-dev mailing list </div></div>> > > > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> > <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>> > > > > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > > > > > > > > > > _______________________________________________ > > > Pulp-dev mailing list > > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> > <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>> > > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > _______________________________________________ > > Pulp-dev mailing list > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> > <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>> > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <div class="HOEnZb"><div class="h5">> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > </div></div> _______________________________________________ Pulp-dev mailing list <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> </blockquote></div> </div>