<div dir="ltr">The only suggestion I'll make is that it would be a good idea to cook up a PoC of un-nesting some of these resources, and make sure it's easy enough to create the appropriate references via a serializer field. For example, right now you can easily use DRF to make a link to "api/v3/repositories/1/publishers/" and include that in the representation of the repository. But in a flat world I suppose you'll need that link to be "api/v3/publishers/?repository=api/v3/repositories/1/" or similar.<div> </div><div>Hopefully it's easy to do with DRF and will be a non-event, but we've been surprised by DNF before. Nesting definitely adds complexity, and flattening is very likely to reduce that code complexity. But it'll be nice to know for sure and by how much.</div></div><div class="gmail_extra"> <div class="gmail_quote">On Thu, Nov 30, 2017 at 1:30 PM, Mihai Ibanescu <<a href="mailto:mihai.ibanescu@gmail.com" target="_blank">mihai.ibanescu@gmail.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">We just happened to have a discussion at work about REST APIs, so I was going to share the benefits of flat URIs.<div> </div><div>If the REST API supports filtering (ours do), then the user may want to search for, let's say, all the distributors whose relative_url starts with "marvin/" in order to perform some operation on the repository.</div><div> </div><div>A filter is typically a GET request with query params on the collection URL. If the distributors collection URL is <a href="https://api/v1/.../distributors/" target="_blank">https://api/v1/.../distributors/</a>, then you'd add ?filter=startswith(config.relative_url,marvin) at the end of the URL - assuming that's the filtering syntax you use.</div><div> </div><div>In a nested world, that's awkward, because the "distributors" collection URL lives under the repository resource, so you'd have to iterate over each repository. Or you'd have to provide a special top-level /api/v1/distributors/ just for searching, but at that point why not just move everything there?</div><div> </div><div>Mihai</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"> <div class="gmail_quote">On Thu, Nov 30, 2017 at 1:11 PM, Daniel Alley <<a href="mailto:dalley@redhat.com" target="_blank">dalley@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">+0 to un-nesting<div><div class="m_-4256564919648668469h5"> <div><div class="gmail_extra"><div class="gmail_quote">On Thu, Nov 30, 2017 at 11:19 AM, Bihan Zhang <<a href="mailto:bizhang@redhat.com" target="_blank">bizhang@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">After chatting with @asmacdo I am now +0 on this. <div>I've been convinced that treating importers, publishers, and content as separate resources is a reasonable approach. </div></div><div class="m_-4256564919648668469m_4837827771909237136m_-7215543513786777089HOEnZb"><div class="m_-4256564919648668469m_4837827771909237136m_-7215543513786777089h5"><div class="gmail_extra"> <div class="gmail_quote">On Thu, Nov 30, 2017 at 10:40 AM, Jeff Ortel <<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">+1 to flattening. On 11/30/2017 08:14 AM, David Davis wrote: > +1 to un-nesting for me as well. > > > David > > On Thu, Nov 30, 2017 at 8:48 AM, Dennis Kliban <<a href="mailto:dkliban@redhat.com" target="_blank">dkliban@redhat.com</a> <mailto:<a href="mailto:dkliban@redhat.com" target="_blank">dkliban@redhat.com</a>>> wrote: > > +1 to not nesting > > I prefer the simplicity of unnested URLs for the API. This change will require users to specify a > repository href when creating an importer or a publisher. This provides the same amount of information as > a nested URL. > > On Wed, Nov 29, 2017 at 5:32 PM, Brian Bouterse <<a href="mailto:bbouters@redhat.com" target="_blank">bbouters@redhat.com</a> <mailto:<a href="mailto:bbouters@redhat.com" target="_blank">bbouters@redhat.com</a>>> wrote: > > For deletes, the db relationships are all there, so I expect deletes to cascade to other objects with > any url structure. I believe closer to the release, we'll have to look at the cascading delete > relationships to see if the behaviors that we have are correct. > > Overall, I'm +1 on un-nesting. I think it would result in a good user experience. I know it goes > against the logical composition arguments, which have been well laid out. We want Pulp to be really > simple, and the nested URL in the top of this thread is anything but simple. Consider another project > like Ansible Galaxy (who also uses Django and DRF). Their API is very flat and as an outsider I find > it very approachable: <a href="https://galaxy.ansible.com/api/v1/" rel="noreferrer" target="_blank">https://galaxy.ansible.com/api/v1/</a> <<a href="https://galaxy.ansible.com/api/v1/" rel="noreferrer" target="_blank">https://galaxy.ansible.com/api/v1/</a>> Pulp > could be that simple. > > My main concern in keeping the nesting is that this is going to be difficult for plugin writers. > Making plugin writing easy is a primary goal if not the primary goal of Pulp3. If core devs are > spending lots of time on it, a person doing this in their free time may not bother. > > I also see practical reasons motivating us to un-nest. We have been adding custom code regularly in > this area, and it's been highly complexity and slow going. I think Austin described it well. Getting > the viewsets working and to be simpler would allow us to move forward in many areas. > > So overall, un-nesting would give a better user experience (I think), a simpler plugin writer > experience, and it would unblock a lot of work. > > > > On Wed, Nov 29, 2017 at 3:29 PM, Bihan Zhang <<a href="mailto:bizhang@redhat.com" target="_blank">bizhang@redhat.com</a> <mailto:<a href="mailto:bizhang@redhat.com" target="_blank">bizhang@redhat.com</a>>> wrote: > > I have a question about repository delete with the un-nested model. > When a repository is deleted does the DELETE cascade to the importers/publishers that are linked > to the repo? In an un-nested world I don't think they would. It would be odd for an object with > its own endpoint to vanish without the user calling DELETE on the model. > > When nested it makes sense to cascade the delete so if /repo/1/ is deleted, everything thereafter > (/repo/1/importer/2) should also be removed. > > Austin, I do see you point about it being a lot more complicated, but I think modeling things the > right way is worth carrying the extra code and complexity. > > Anyways, maybe I'm wrong and importer/publishers should exist without a repository, in which case > I can definitely see the value in un-nesting the URLs. > > <div><div class="m_-4256564919648668469m_4837827771909237136m_-7215543513786777089m_1958130310844086241h5">> On Wed, Nov 29, 2017 at 2:21 PM, Jeff Ortel <<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a> <mailto:<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>>> wrote: > > Austin makes a compelling argument. > > > On 11/28/2017 02:16 PM, Austin Macdonald wrote: > > When I look at this, the most important point is that we have a hyperlinked REST API, which > means that the > > urls are specifically not going to be built by users. > > > > For a user to retrieve an importer, they would first GET the importers for a repository. The > next call would > > be the exact href returned by pulp. This workflow is exactly the same whether we nest or > not. The only > > difference is that we no longer convey the information in the href, which seems fine to me > since they aren't > > particularly readable anyway. > > > > It has already been discussed that filtering can make up for the use cases that use nesting, > and that filters > > would be more flexible. > > > > So for me, nesting costs in (1) extra code to carry (2) extra dependency (3) complexity to use. > > > > To elaborate on the complexity, the problem is in declaring fields on the serializer. The > serializer is > > responsible for building the urls, which requires all of the uuids for the entire nested > structure. This is > > further complicated by master/detail, which is an entirely Pulp concept. > > > > Because of this, anyone working on the API (likely including plugin writers) will need to > understand > > parent_lookup_kwargs and how to use then with: > > DetailNestedHyperlinkedRelatedField > > DetailNestedHyperlinkedidentityField > > DetailwritableNestedUrlRelatedField > > DetailRelatedField > > DetailIdentityField > > NestedHyperlinkedRelatedField > > HyperlinkedRelatedField. > > > > The complexity seems inherrent, so I doubt we will be able to simplify this much. So, is all > this code and > > complexity worth the implied relationship in non-human-friendly urls? As someone who has > spent a lot of time > > on this code, I don't think so. > > > > > > > > On Nov 28, 2017 06:12, "Patrick Creech" <<a href="mailto:pcreech@redhat.com" target="_blank">pcreech@redhat.com</a> <mailto:<a href="mailto:pcreech@redhat.com" target="_blank">pcreech@redhat.com</a>> </div></div><div><div class="m_-4256564919648668469m_4837827771909237136m_-7215543513786777089m_1958130310844086241h5">> <mailto:<a href="mailto:pcreech@redhat.com" target="_blank">pcreech@redhat.com</a> <mailto:<a href="mailto:pcreech@redhat.com" target="_blank">pcreech@redhat.com</a>>>> wrote: > > > > On Mon, 2017-11-27 at 16:10 -0600, Jeff Ortel wrote: > > > On 11/27/2017 12:19 PM, Jeff Ortel wrote: > > > > > > > > > > > > On 11/17/2017 08:55 AM, Patrick Creech wrote: > > > > > One of the things I like to think about in these types of situations is, "what is > good rest > > > > > api > > > > > design". Nesting resources under other resources is a necessary part of good api > design, and > > > > > has > > > > > its place. To borrow some terms from domain driven development: > > > > > > > > > > Collections of objects are called aggregates. Think 'an order and its line > items'. Line > > > > > items make > > > > > no sense without having the order context, so they are an aggregate that is > accessed under an > > > > > Order. This is called the aggregate root. The rest api design for such an > object, using > > > > > order as > > > > > the aggregate root, would look like: > > > > > > > > > > '/orders/' -- all orders > > > > > '/orders/{order_key}/' -- a specific order with key. > > > > > '/orders/{order_key}/items/' -- All of the order's items. > > > > > '/orders/{order_key}/items/{item_key}/' -- a specific line item of the order > > > > > > > > > > When it comes to order items themselves, it isn't helpful to start with them as > their own > > > > > aggregate > > > > > root in one large collection: > > > > > > > > > > '/items/' -- all order items in the system > > > > > > > > The order/items is a good example of aggregation (or composition) and I agree it > makes a strong > > > > case for > > > > nesting. In pulp, a repository is easily thought of as a collection or aggregation > of content. > > > > > > > > > > > > > > Because you lose the order context. Based on api design, this endpoint will need > to respond > > > > > with all > > > > > order items across all orders and resort to parameter filtering to provide the > context you > > > > > need. > > > > > > > > > > A quote borrowed from Martin Fowler [0] > > > > > > > > > > "An aggregate will have one of its component objects be the aggregate root. Any > references > > > > > from > > > > > outside the aggregate should only go to the aggregate root. The root can thus > ensure the > > > > > integrity > > > > > of the aggregate as a whole." > > > > > > > > > > Publishers, importers, and publications are all aggregates that don't make much > sense outside > > > > > of > > > > > their aggregate root of Repository. They are dependent on the Repository context, > and from a > > > > > domain > > > > > view, should be accessed starting with their specific Repository endpoint. > > > > > > > > I don't think the aggregation relationship exists between repository and > > > > importer/publisher. There is a > > > > strong association between repository and importer/publisher which /could/ even be > characterized > > > > as > > > > "ownership". However, I don't think there is an aggregation (or composition) > relationship. The > > > > same for > > > > publisher & publication. A publication is associated to its creating publisher but the > > > > publisher isn't an > > > > aggregation of publications. The relationship mainly provides linkage to the > repository. > > > > > > This is not an argument to flatten the URLs but meant to clarify the relationships. > > > > I'm in agreement here. I was possibly a little hasty in lumping all things that have a > Repositoy fk > > as being 'dependent' in that paragraph during the formation of my argument. > > > > > > > > > > > > > > > > -------------------------------------------------------------- > > > > > Specific items rebuttals: > > > > > > > > > > Yes, using the primary key uuid's as the immutable key adds some human > readable challenges > > > > > to > > > > > the API. That sounds more like a point to discuss in the human readable vs. not human > > > > > readable > > > > > immutable key debate. > > > > > > > > Agreed. > > > > > > > > Also, I don't think nesting impacts URL readability. > > > > > > > > > > > > > > One of the challenges in software engineering is ensuring the tools you are > using don't > > > > > limit > > > > > your choices. DRF limited the choices for pulp's rest API design, and > drf-nested-routers was > > > > > introduced to help remove that limit. If working around these limitations is > complex, take > > > > > advantage of open source here and help improve the upstream dependencies for your > workflow. > > > > > > > > > > As far as making things simpler for plugin writers, perhaps there are ways you can > > > > > simplify it > > > > > for them by providing some encapsulation in pulp's core instead. Abstract away > the nasty bits > > > > > behind the scenes, and provide them with a simpler interface to do what they need. > > > > > > > > > > With respect to the invested time already in making this work, I agree with > jeremy that it > > > > > should be considered part of the sunken cost fallacy. What does need to be > evaluated though > > > > > is how > > > > > much time re-architecting at this point will cost you (discussion, planning, and > development) > > > > > vs the > > > > > amount of time it will save, and weigh that against any planned milestones for > pulp to see if > > > > > it > > > > > will push them out as well. > > > > > > > > > > I'm also in agreement that it is moot if pulp3 has a different api structure than > > > > > pulp2. Major > > > > > version boundaries are the perfect time for evaluating and moving such things around. > > > > > > > > > > [0] <a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a> > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a>> > > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a> > <<a href="https://martinfowler.com/bliki/DDD_Aggregate.html" rel="noreferrer" target="_blank">https://martinfowler.com/bliki/DDD_Aggregate.html</a>>> > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Pulp-dev mailing list </div></div>> > > > > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> > <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>>> > > > > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > > > > > > > > > > _______________________________________________ > > > Pulp-dev mailing list > > > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> > <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>>> > > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > _______________________________________________ > > Pulp-dev mailing list > > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <div class="m_-4256564919648668469m_4837827771909237136m_-7215543513786777089m_1958130310844086241HOEnZb"><div class="m_-4256564919648668469m_4837827771909237136m_-7215543513786777089m_1958130310844086241h5">> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>>> > > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>>> > > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > </div></div> _______________________________________________ Pulp-dev mailing list <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> </blockquote></div> </div> </div></div> _______________________________________________ Pulp-dev mailing list <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> </blockquote></div> </div></div></div></div></div> _______________________________________________ Pulp-dev mailing list <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> </blockquote></div> </div> </div></div> _______________________________________________ Pulp-dev mailing list <a href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> </blockquote></div> <div> </div>-- <div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Michael HrivnakPrincipal Software Engineer, RHCE Red Hat</div></div> </div>