<div dir="ltr">After chatting with @asmacdo I am now +0 on this. <div>I've been convinced that treating importers, publishers, and content as separate resources is a reasonable approach. </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Nov 30, 2017 at 10:40 AM, Jeff Ortel <span dir="ltr"><<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">+1 to flattening.<br>
<span class=""><br>
On 11/30/2017 08:14 AM, David Davis wrote:<br>
> +1 to un-nesting for me as well.<br>
><br>
><br>
> David<br>
><br>
</span><span class="">> On Thu, Nov 30, 2017 at 8:48 AM, Dennis Kliban <<a href="mailto:dkliban@redhat.com">dkliban@redhat.com</a> <mailto:<a href="mailto:dkliban@redhat.com">dkliban@redhat.com</a>>> wrote:<br>
><br>
> +1 to not nesting<br>
><br>
> I prefer the simplicity of unnested URLs for the API. This change will require users to specify a<br>
> repository href when creating an importer or a publisher. This provides the same amount of information as<br>
> a nested URL.<br>
><br>
</span><span class="">> On Wed, Nov 29, 2017 at 5:32 PM, Brian Bouterse <<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com">bbouters@redhat.com</a>>> wrote:<br>
><br>
> For deletes, the db relationships are all there, so I expect deletes to cascade to other objects with<br>
> any url structure. I believe closer to the release, we'll have to look at the cascading delete<br>
> relationships to see if the behaviors that we have are correct.<br>
><br>
> Overall, I'm +1 on un-nesting. I think it would result in a good user experience. I know it goes<br>
> against the logical composition arguments, which have been well laid out. We want Pulp to be really<br>
> simple, and the nested URL in the top of this thread is anything but simple. Consider another project<br>
> like Ansible Galaxy (who also uses Django and DRF). Their API is very flat and as an outsider I find<br>
</span>> it very approachable: <a href="https://galaxy.ansible.com/api/v1/" rel="noreferrer" target="_blank">https://galaxy.ansible.com/<wbr>api/v1/</a> <<a href="https://galaxy.ansible.com/api/v1/" rel="noreferrer" target="_blank">https://galaxy.ansible.com/<wbr>api/v1/</a>> Pulp<br>
<span class="">> could be that simple.<br>
><br>
> My main concern in keeping the nesting is that this is going to be difficult for plugin writers.<br>
> Making plugin writing easy is a primary goal if not the primary goal of Pulp3. If core devs are<br>
> spending lots of time on it, a person doing this in their free time may not bother.<br>
><br>
> I also see practical reasons motivating us to un-nest. We have been adding custom code regularly in<br>
> this area, and it's been highly complexity and slow going. I think Austin described it well. Getting<br>
> the viewsets working and to be simpler would allow us to move forward in many areas.<br>
><br>
> So overall, un-nesting would give a better user experience (I think), a simpler plugin writer<br>
> experience, and it would unblock a lot of work.<br>
><br>
><br>
><br>
</span><span class="">> On Wed, Nov 29, 2017 at 3:29 PM, Bihan Zhang <<a href="mailto:bizhang@redhat.com">bizhang@redhat.com</a> <mailto:<a href="mailto:bizhang@redhat.com">bizhang@redhat.com</a>>> wrote:<br>
><br>
> I have a question about repository delete with the un-nested model.<br>
> When a repository is deleted does the DELETE cascade to the importers/publishers that are linked<br>
> to the repo? In an un-nested world I don't think they would. It would be odd for an object with<br>
> its own endpoint to vanish without the user calling DELETE on the model.<br>
><br>
> When nested it makes sense to cascade the delete so if /repo/1/ is deleted, everything thereafter<br>
> (/repo/1/importer/2) should also be removed.<br>
><br>
> Austin, I do see you point about it being a lot more complicated, but I think modeling things the<br>
> right way is worth carrying the extra code and complexity.<br>
><br>
> Anyways, maybe I'm wrong and importer/publishers should exist without a repository, in which case<br>
> I can definitely see the value in un-nesting the URLs.<br>
><br>
><br>
</span><div><div class="h5">> On Wed, Nov 29, 2017 at 2:21 PM, Jeff Ortel <<a href="mailto:jortel@redhat.com">jortel@redhat.com</a> <mailto:<a href="mailto:jortel@redhat.com">jortel@redhat.com</a>>> wrote:<br>
><br>
> Austin makes a compelling argument.<br>
><br>
><br>
> On 11/28/2017 02:16 PM, Austin Macdonald wrote:<br>
> > When I look at this, the most important point is that we have a hyperlinked REST API, which<br>
> means that the<br>
> > urls are specifically not going to be built by users.<br>
> ><br>
> > For a user to retrieve an importer, they would first GET the importers for a repository. The<br>
> next call would<br>
> > be the exact href returned by pulp. This workflow is exactly the same whether we nest or<br>
> not. The only<br>
> > difference is that we no longer convey the information in the href, which seems fine to me<br>
> since they aren't<br>
> > particularly readable anyway.<br>
> ><br>
> > It has already been discussed that filtering can make up for the use cases that use nesting,<br>
> and that filters<br>
> > would be more flexible.<br>
> ><br>
> > So for me, nesting costs in (1) extra code to carry (2) extra dependency (3) complexity to use.<br>
> ><br>
> > To elaborate on the complexity, the problem is in declaring fields on the serializer. The<br>
> serializer is<br>
> > responsible for building the urls, which requires all of the uuids for the entire nested<br>
> structure. This is<br>
> > further complicated by master/detail, which is an entirely Pulp concept.<br>
> ><br>
> > Because of this, anyone working on the API (likely including plugin writers) will need to<br>
> understand<br>
> > parent_lookup_kwargs and how to use then with:<br>
> > DetailNestedHyperlinkedRelated<wbr>Field<br>
> > DetailNestedHyperlinkedidentit<wbr>yField<br>
> > DetailwritableNestedUrlRelated<wbr>Field<br>
> > DetailRelatedField<br>
> > DetailIdentityField<br>
> > NestedHyperlinkedRelatedField<br>
> > HyperlinkedRelatedField.<br>
> ><br>
> > The complexity seems inherrent, so I doubt we will be able to simplify this much. So, is all<br>
> this code and<br>
> > complexity worth the implied relationship in non-human-friendly urls? As someone who has<br>
> spent a lot of time<br>
> > on this code, I don't think so.<br>
> ><br>
> ><br>
> ><br>
> > On Nov 28, 2017 06:12, "Patrick Creech" <<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a> <mailto:<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a>><br>
</div></div><div><div class="h5">> <mailto:<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a> <mailto:<a href="mailto:pcreech@redhat.com">pcreech@redhat.com</a>>>> wrote:<br>
> ><br>
> > On Mon, 2017-11-27 at 16:10 -0600, Jeff Ortel wrote:<br>
> > > On 11/27/2017 12:19 PM, Jeff Ortel wrote:<br>
> > > ><br>
> > > ><br>
> > > > On 11/17/2017 08:55 AM, Patrick Creech wrote:<br>
> > > > > One of the things I like to think about in these types of situations is, "what is<br>
> good rest<br>
> > > > > api<br>
> > > > > design". Nesting resources under other resources is a necessary part of good api<br>
> design, and<br>
> > > > > has<br>
> > > > > its place. To borrow some terms from domain driven development:<br>
> > > > ><br>
> > > > > Collections of objects are called aggregates. Think 'an order and its line<br>
> items'. Line<br>
> > > > > items make<br>
> > > > > no sense without having the order context, so they are an aggregate that is<br>
> accessed under an<br>
> > > > > Order. This is called the aggregate root. The rest api design for such an<br>
> object, using<br>
> > > > > order as<br>
> > > > > the aggregate root, would look like:<br>
> > > > ><br>
> > > > > '/orders/' -- all orders<br>
> > > > > '/orders/{order_key}/' -- a specific order with key.<br>
> > > > > '/orders/{order_key}/items/' -- All of the order's items.<br>
> > > > > '/orders/{order_key}/items/{<wbr>item_key}/' -- a specific line item of the order<br>
> > > > ><br>
> > > > > When it comes to order items themselves, it isn't helpful to start with them as<br>
> their own<br>
> > > > > aggregate<br>
> > > > > root in one large collection:<br>
> > > > ><br>
> > > > > '/items/' -- all order items in the system<br>
> > > ><br>
> > > > The order/items is a good example of aggregation (or composition) and I agree it<br>
> makes a strong<br>
> > > > case for<br>
> > > > nesting. In pulp, a repository is easily thought of as a collection or aggregation<br>
> of content.<br>
> > > ><br>
> > > > ><br>
> > > > > Because you lose the order context. Based on api design, this endpoint will need<br>
> to respond<br>
> > > > > with all<br>
> > > > > order items across all orders and resort to parameter filtering to provide the<br>
> context you<br>
> > > > > need.<br>
> > > > ><br>
> > > > > A quote borrowed from Martin Fowler [0]<br>
> > > > ><br>
> > > > > "An aggregate will have one of its component objects be the aggregate root. Any<br>
> references<br>
> > > > > from<br>
> > > > > outside the aggregate should only go to the aggregate root. The root can thus<br>
> ensure the<br>
> > > > > integrity<br>
> > > > > of the aggregate as a whole."<br>
> > > > ><br>
> > > > > Publishers, importers, and publications are all aggregates that don't make much<br>
> sense outside<br>
> > > > > of<br>
> > > > > their aggregate root of Repository. They are dependent on the Repository context,<br>
> and from a<br>
> > > > > domain<br>
> > > > > view, should be accessed starting with their specific Repository endpoint.<br>
> > > ><br>
> > > > I don't think the aggregation relationship exists between repository and<br>
> > > > importer/publisher. There is a<br>
> > > > strong association between repository and importer/publisher which /could/ even be<br>
> characterized<br>
> > > > as<br>
> > > > "ownership". However, I don't think there is an aggregation (or composition)<br>
> relationship. The<br>
> > > > same for<br>
> > > > publisher & publication. A publication is associated to its creating publisher but the<br>
> > > > publisher isn't an<br>
> > > > aggregation of publications. The relationship mainly provides linkage to the<br>
> repository.<br>
> > ><br>
> > > This is not an argument to flatten the URLs but meant to clarify the relationships.<br>
> ><br>
> > I'm in agreement here. I was possibly a little hasty in lumping all things that have a<br>
> Repositoy fk<br>
> > as being 'dependent' in that paragraph during the formation of my argument.<br>
> ><br>
> > > ><br>
> > > > ><br>
> > > > > ------------------------------<wbr>------------------------------<wbr>--<br>
> > > > > Specific items rebuttals:<br>
> > > > ><br>
> > > > > Yes, using the primary key uuid's as the immutable key adds some human<br>
> readable challenges<br>
> > > > > to<br>
> > > > > the API. That sounds more like a point to discuss in the human readable vs. not human<br>
> > > > > readable<br>
> > > > > immutable key debate.<br>
> > > ><br>
> > > > Agreed.<br>
> > > ><br>
> > > > Also, I don't think nesting impacts URL readability.<br>
> > > ><br>
> > > > ><br>
> > > > > One of the challenges in software engineering is ensuring the tools you are<br>
> using don't<br>
> > > > > limit<br>
> > > > > your choices. DRF limited the choices for pulp's rest API design, and<br>
> drf-nested-routers was<br>
> > > > > introduced to help remove that limit. If working around these limitations is<br>
> complex, take<br>
> > > > > advantage of open source here and help improve the upstream dependencies for your<br>
> workflow.<br>
> > > > ><br>
> > > > > As far as making things simpler for plugin writers, perhaps there are ways you can<br>
> > > > > simplify it<br>
> > > > > for them by providing some encapsulation in pulp's core instead. Abstract away<br>
> the nasty bits<br>
> > > > > behind the scenes, and provide them with a simpler interface to do what they need.<br>
> > > > ><br>
> > > > > With respect to the invested time already in making this work, I agree with<br>
> jeremy that it<br>
> > > > > should be considered part of the sunken cost fallacy. What does need to be<br>
> evaluated though<br>
> > > > > is how<br>
> > > > > much time re-architecting at this point will cost you (discussion, planning, and<br>
> development)<br>
> > > > > vs the<br>
> > > > > amount of time it will save, and weigh that against any planned milestones for<br>
> pulp to see if<br>
> > > > > it<br>
> > > > > will push them out as well.<br>
> > > > ><br>
> > > > > I'm also in agreement that it is moot if pulp3 has a different api structure than<br>
> > > > > pulp2. Major<br>
> > > > > version boundaries are the perfect time for evaluating and moving such things around.<br>
> > > > ><br>
> > > > > [0] <a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/<wbr>bliki/DDD_Aggregate.html</a><br>
> <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/<wbr>bliki/DDD_Aggregate.html</a>><br>
> > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/<wbr>bliki/DDD_Aggregate.html</a><br>
> <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/<wbr>bliki/DDD_Aggregate.html</a>>><br>
> > > > ><br>
> > > > ><br>
> > > > ><br>
> > > > > ______________________________<wbr>_________________<br>
> > > > > Pulp-dev mailing list<br>
</div></div>> > > > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a><br>
<span class="">> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>><br>
> > > > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>>><br>
> > > > ><br>
> > ><br>
> > > ______________________________<wbr>_________________<br>
> > > Pulp-dev mailing list<br>
</span>> > > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a><br>
<span class="">> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>><br>
> > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>>><br>
> > ______________________________<wbr>_________________<br>
> > Pulp-dev mailing list<br>
</span>> > <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a><br>
<div class="HOEnZb"><div class="h5">> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>>><br>
> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>>><br>
> ><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Pulp-dev mailing list<br>
> <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>><br>
> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Pulp-dev mailing list<br>
> <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>><br>
> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Pulp-dev mailing list<br>
> <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>><br>
> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Pulp-dev mailing list<br>
> <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>><br>
> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a>><br>
><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> Pulp-dev mailing list<br>
> <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a><br>
> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
><br>
<br>
</div></div><br>______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/<wbr>mailman/listinfo/pulp-dev</a><br>
<br></blockquote></div><br></div>