From lutter at redhat.com Mon May 3 21:51:09 2010 From: lutter at redhat.com (David Lutterkort) Date: Mon, 03 May 2010 14:51:09 -0700 Subject: [rest-practices] Namespace qualifying respresentation types In-Reply-To: <1272639984.3698.329.camel@quark> References: <20100430141924.GK32184@redhat.com> <1272639984.3698.329.camel@quark> Message-ID: <1272923469.14660.16.camel@localhost> On Fri, 2010-04-30 at 16:06 +0100, Eoghan Glynn wrote: > On Fri, 2010-04-30 at 16:19 +0200, Daniel Veillard wrote: > > On Thu, Apr 29, 2010 at 11:55:07AM +0100, Eoghan Glynn wrote: > > > Folks, > > > > > > I wanted to get the group's feeling for the use of namespaces in > > > representation types. > > > > > > The context for the question initially comes from a discussion about how > > > action/operations were to be modelled in RHEVM-API, particularly how to > > > capture the action-specific parameter types in the schema. In XML schema > > > terms, my first cut was to use an xs:any within the action element, but to > > > restrict the contained type to the local namespace. Which of course requires > > > that a target namespace is defined in the first place. > > > > Why XML Schemas ? Why not Relax-NG instead. > > > Yeah, I'd love to learn RELAX NG properly. > > However on the RHEVM-API project we've codegen'd the JAXB model from > schema, and unfortunately the JAXB compiler's support for RNG is marked > "(experimental,unsupported)" so I'd be a bit nervous about relying on > it. Trang[1] will convert from rng (or rnc) to xsd. And it's written in java. David [1] http://www.thaiopensource.com/relaxng/trang.html From markmc at redhat.com Tue May 4 07:24:18 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Tue, 04 May 2010 08:24:18 +0100 Subject: [rest-practices] Namespace qualifying respresentation types In-Reply-To: <1272923469.14660.16.camel@localhost> References: <20100430141924.GK32184@redhat.com> <1272639984.3698.329.camel@quark> <1272923469.14660.16.camel@localhost> Message-ID: <1272957858.2908.10.camel@blaa> On Mon, 2010-05-03 at 14:51 -0700, David Lutterkort wrote: > On Fri, 2010-04-30 at 16:06 +0100, Eoghan Glynn wrote: > > On Fri, 2010-04-30 at 16:19 +0200, Daniel Veillard wrote: > > > On Thu, Apr 29, 2010 at 11:55:07AM +0100, Eoghan Glynn wrote: > > > > Folks, > > > > > > > > I wanted to get the group's feeling for the use of namespaces in > > > > representation types. > > > > > > > > The context for the question initially comes from a discussion about how > > > > action/operations were to be modelled in RHEVM-API, particularly how to > > > > capture the action-specific parameter types in the schema. In XML schema > > > > terms, my first cut was to use an xs:any within the action element, but to > > > > restrict the contained type to the local namespace. Which of course requires > > > > that a target namespace is defined in the first place. > > > > > > Why XML Schemas ? Why not Relax-NG instead. > > > > > > Yeah, I'd love to learn RELAX NG properly. > > > > However on the RHEVM-API project we've codegen'd the JAXB model from > > schema, and unfortunately the JAXB compiler's support for RNG is marked > > "(experimental,unsupported)" so I'd be a bit nervous about relying on > > it. > > Trang[1] will convert from rng (or rnc) to xsd. And it's written in > java. Yeah, we used that already - I'm going to play around with adding it to our build process so we can use RNG The reason being that rather than having: we could have: NFS ISCSI which much more clearly shows the relationship between the 'type' attribute and the content Cheers, Mark. From veillard at redhat.com Tue May 4 09:00:50 2010 From: veillard at redhat.com (Daniel Veillard) Date: Tue, 4 May 2010 11:00:50 +0200 Subject: [rest-practices] Namespace qualifying respresentation types In-Reply-To: <1272923469.14660.16.camel@localhost> References: <20100430141924.GK32184@redhat.com> <1272639984.3698.329.camel@quark> <1272923469.14660.16.camel@localhost> Message-ID: <20100504090050.GC32184@redhat.com> On Mon, May 03, 2010 at 02:51:09PM -0700, David Lutterkort wrote: > On Fri, 2010-04-30 at 16:06 +0100, Eoghan Glynn wrote: > > On Fri, 2010-04-30 at 16:19 +0200, Daniel Veillard wrote: > > > On Thu, Apr 29, 2010 at 11:55:07AM +0100, Eoghan Glynn wrote: > > > > Folks, > > > > > > > > I wanted to get the group's feeling for the use of namespaces in > > > > representation types. > > > > > > > > The context for the question initially comes from a discussion about how > > > > action/operations were to be modelled in RHEVM-API, particularly how to > > > > capture the action-specific parameter types in the schema. In XML schema > > > > terms, my first cut was to use an xs:any within the action element, but to > > > > restrict the contained type to the local namespace. Which of course requires > > > > that a target namespace is defined in the first place. > > > > > > Why XML Schemas ? Why not Relax-NG instead. > > > > > > Yeah, I'd love to learn RELAX NG properly. > > > > However on the RHEVM-API project we've codegen'd the JAXB model from > > schema, and unfortunately the JAXB compiler's support for RNG is marked > > "(experimental,unsupported)" so I'd be a bit nervous about relying on > > it. > > Trang[1] will convert from rng (or rnc) to xsd. And it's written in > java. When possible, the expressiveness of rng is far superior to xsd. But I assume for most binding generation that should not be a problem. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel at veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ From markmc at redhat.com Tue May 4 13:19:02 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Tue, 04 May 2010 14:19:02 +0100 Subject: [rest-practices] Modelling membership Message-ID: <1272979142.2908.99.camel@blaa> Hey, We're just hitting something in rhevm-api which I think is fairly generic. So, I'll use a simple user/groups example instead of a RHEV specific example. A user may be a member of multiple groups. We have a top-level /users collection and a top-level /groups collection. How do we modelling adding a user to a group? 1) a element under One thing we didn't like about this is race conditions when editing the membership list using PUT - e.g. A and B see list as { bob, bill, bryan }, client A adds { eoghan } and client B adds { mark }. The result is dependent on which client posts last Eoghan suggests investigating whether a client could specify Etag in its PUT to mean "only accept the update if the current content matches this etag". If it fails, the client would re-GET, re-do its edit and re-PUT. 2) a /users/ collection under a group e.g. Adding a user would be: POST /group/1234/users HTTP/1.1 201 Created Location: /group/1234/users/5678 We liked this because we wanted a per-group user URI. See[1] It also gave a nice solution to the concurrency issue above. We didn't like the fact that it was ambiguous whether the POST should create a new user or just add an existing user to the group's users collection. Also, in general, there is less expressiveness possible with POSTing to a collection rather than adding to a list - e.g. you can't order the elements in the collection with POST. Any thoughts? Thanks, Mark. [1] - this is where things get messy. Instead of "group" read "data center" and instead of "user" read "storage domain" A storage domain can be attach to multiple data centers. In the current RHEV-M API, a storage domain's state is dependent on which data center you are viewing it from e.g. a storage domain can be "attached" in one data center and "active" in another data center. If you query a storage domain outside of the context of a data center its status is "mixed" So, we're modelling that as: /datacenters/1111/storagedomains/5678 state = "attached" /datacenters/2222/storagedomains/5678 state = "active" /storagedomains/5678 state = "mixed" From bmcwhirt at redhat.com Tue May 4 13:28:49 2010 From: bmcwhirt at redhat.com (Bob McWhirter) Date: Tue, 4 May 2010 09:28:49 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <1272979142.2908.99.camel@blaa> References: <1272979142.2908.99.camel@blaa> Message-ID: 3rd resource: "membership" Sometimes called an association object. In our model, I'm sure we end up with code to pair a User to a Group. Ta-da! Expose as REST. Adjusting memberships now only affect the membership object you add or delete, no race conditions of updating a "list" or "roster" from either User or Group end of things. I'd probably veer towards Membership being a top-level object also, owned by neither User nor Group, but linked to from either, as a collection of pointers to the actual Membership resources. -Bob On May 4, 2010, at 9:19 AM, Mark McLoughlin wrote: > Hey, > > We're just hitting something in rhevm-api which I think is fairly > generic. So, I'll use a simple user/groups example instead of a RHEV > specific example. > > A user may be a member of multiple groups. We have a top-level /users > collection and a top-level /groups collection. How do we modelling > adding a user to a group? > > 1) a element under > > One thing we didn't like about this is race conditions when editing > the membership list using PUT - e.g. A and B see list as { bob, > bill, bryan }, client A adds { eoghan } and client B adds { mark }. > The result is dependent on which client posts last > > Eoghan suggests investigating whether a client could specify Etag > in its PUT to mean "only accept the update if the current content > matches this etag". If it fails, the client would re-GET, re-do its > edit and re-PUT. > > 2) a /users/ collection under a group > > e.g. > > Adding a user would be: > > POST /group/1234/users > > > > HTTP/1.1 201 Created > Location: /group/1234/users/5678 > > We liked this because we wanted a per-group user URI. See[1] > > It also gave a nice solution to the concurrency issue above. > > We didn't like the fact that it was ambiguous whether the POST > should create a new user or just add an existing user to the > group's users collection. > > Also, in general, there is less expressiveness possible with > POSTing to a collection rather than adding to a list - e.g. you > can't order the elements in the collection with POST. > > Any thoughts? > > Thanks, > Mark. > > [1] - this is where things get messy. Instead of "group" read "data > center" and instead of "user" read "storage domain" > > A storage domain can be attach to multiple data centers. In the > current > RHEV-M API, a storage domain's state is dependent on which data center > you are viewing it from > > e.g. a storage domain can be "attached" in one data center and > "active" > in another data center. If you query a storage domain outside of the > context of a data center its status is "mixed" > > So, we're modelling that as: > > /datacenters/1111/storagedomains/5678 state = "attached" > /datacenters/2222/storagedomains/5678 state = "active" > /storagedomains/5678 state = "mixed" > > _______________________________________________ > rest-practices mailing list > rest-practices at redhat.com > https://www.redhat.com/mailman/listinfo/rest-practices From markmc at redhat.com Tue May 4 13:31:00 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Tue, 04 May 2010 14:31:00 +0100 Subject: [rest-practices] [Fwd: [rhevm-api] concatenating Link headers in response to entry-point HEAD method] Message-ID: <1272979860.2908.103.camel@blaa> Hey, I thought this had been discussed here, but I don't think so. Eoghan found some issues with including multiple Link: headers. I've seen something similar - a ruby client was only giving me a single Link: header. We've decided to concatenate our Link headers together on the server side, as ugly as that is. Cheers, Mark. -------- Forwarded Message -------- > From: Eoghan Glynn > To: rhevm-api > Subject: [rhevm-api] concatenating Link headers in response to > entry-point HEAD method > Date: Mon, 26 Apr 2010 09:11:24 +0100 > > > Folks, > > We've noticed some funnies with the handling of multiple headers of the > same name by various REST providers, with RESTeasy concatenating the > header values into a comma separated list, and CXF JAX-RS seemingly > dropping all but the last header instance. > > In the light of the following chapter & verse from the spec: > > "Multiple message-header fields with the same field-name MAY be present > in a message if and only if the entire field-value for that header field > is defined as a comma-separated list [i.e., #(values)]. It MUST be > possible to combine the multiple header fields into one "field-name: > field-value" pair, without changing the semantics of the message, by > appending each subsequent field-value to the first, each separated by a > comma. The order in which header fields with the same field-name are > received is therefore significant to the interpretation of the combined > field value, and thus a proxy MUST NOT change the order of these field > values when a message is forwarded."[1] > > It seems like a concatenation is the canonical form and least likely > cause trouble along the way. So how about we simply squash together the > Links set in the implementations of ApiResource.head()? > > Cheers, > Eoghan > > > [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 > > > > _______________________________________________ > rhevm-api mailing list > rhevm-api at lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/rhevm-api From markmc at redhat.com Tue May 4 14:15:39 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Tue, 04 May 2010 15:15:39 +0100 Subject: [rest-practices] Modelling membership In-Reply-To: References: <1272979142.2908.99.camel@blaa> Message-ID: <1272982539.2908.111.camel@blaa> On Tue, 2010-05-04 at 09:28 -0400, Bob McWhirter wrote: > 3rd resource: "membership" > > Sometimes called an association object. > > In our model, I'm sure we end up with code to pair a User to a Group. > Ta-da! Expose as REST. Adjusting memberships now only affect the > membership object you add or delete, no race conditions of updating a > "list" or "roster" from either User or Group end of things. > > I'd probably veer towards Membership being a top-level object also, > owned by neither User nor Group, but linked to from either, as a > collection of pointers to the actual Membership resources. Nice idea, thanks We'll probably do this as a collection of "attachment" resources owned by a storage domain - each attachment just includes a reference to a data center and the status of the storage domain within that data center I'd prefer this approach to having it toplevel, since it means one side of the relationship is implicit Cheers, Mark. From bkearney at redhat.com Tue May 4 16:21:19 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Tue, 04 May 2010 12:21:19 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <1272982539.2908.111.camel@blaa> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> Message-ID: <4BE0497F.4020103@redhat.com> On 05/04/2010 10:15 AM, Mark McLoughlin wrote: > On Tue, 2010-05-04 at 09:28 -0400, Bob McWhirter wrote: >> 3rd resource: "membership" >> >> Sometimes called an association object. >> >> In our model, I'm sure we end up with code to pair a User to a Group. >> Ta-da! Expose as REST. Adjusting memberships now only affect the >> membership object you add or delete, no race conditions of updating a >> "list" or "roster" from either User or Group end of things. >> >> I'd probably veer towards Membership being a top-level object also, >> owned by neither User nor Group, but linked to from either, as a >> collection of pointers to the actual Membership resources. > > Nice idea, thanks > > We'll probably do this as a collection of "attachment" resources owned > by a storage domain - each attachment just includes a reference to a > data center and the status of the storage domain within that data center > > I'd prefer this approach to having it toplevel, since it means one side > of the relationship is implicit > I like the idea, but if you make it under a top level item, doesn't this degrade to /groups/{uid}/user? -- bk From markmc at redhat.com Tue May 4 16:29:55 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Tue, 04 May 2010 17:29:55 +0100 Subject: [rest-practices] Modelling membership In-Reply-To: <4BE0497F.4020103@redhat.com> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> Message-ID: <1272990595.2908.164.camel@blaa> On Tue, 2010-05-04 at 12:21 -0400, Bryan Kearney wrote: > On 05/04/2010 10:15 AM, Mark McLoughlin wrote: > > On Tue, 2010-05-04 at 09:28 -0400, Bob McWhirter wrote: > >> 3rd resource: "membership" > >> > >> Sometimes called an association object. > >> > >> In our model, I'm sure we end up with code to pair a User to a Group. > >> Ta-da! Expose as REST. Adjusting memberships now only affect the > >> membership object you add or delete, no race conditions of updating a > >> "list" or "roster" from either User or Group end of things. > >> > >> I'd probably veer towards Membership being a top-level object also, > >> owned by neither User nor Group, but linked to from either, as a > >> collection of pointers to the actual Membership resources. > > > > Nice idea, thanks > > > > We'll probably do this as a collection of "attachment" resources owned > > by a storage domain - each attachment just includes a reference to a > > data center and the status of the storage domain within that data center > > > > I'd prefer this approach to having it toplevel, since it means one side > > of the relationship is implicit > > > > I like the idea, but if you make it under a top level item, doesn't this > degrade to /groups/{uid}/user? By toplevel, I mean "blaa.com/memberships" So, by not having it toplevel, what we're thinking is equivalent to: POST /users/1234/memberships HTTP/1.1 200 OK Location: /users/1234/memberships/1111 We'll also make the membership list available read-only via the group: GET /groups/4321 foo ... ... Cheers, Mark. From bmcwhirt at redhat.com Tue May 4 17:00:06 2010 From: bmcwhirt at redhat.com (Bob McWhirter) Date: Tue, 4 May 2010 13:00:06 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <1272990595.2908.164.camel@blaa> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> Message-ID: <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> > So, by not having it toplevel, what we're thinking is equivalent to: > > POST /users/1234/memberships > > > > Generally speaking, I'd rather see hrefs instead of IDs: I don't like this partial in-lined description of a resource, though, as then I have to keep up with user_id separately, it's not a part of the representation when fetched this way. It's easier to code if I know I can resolve a URL to fetch a complete object, instead of having to considering it within some context. Particularly if we're ignoring URL structure and being HATEOASishy. Depending on how "pure" you wanted to be, I might also do And treat it as a collection of references. This is what I mean by a "top-level" object. One that is directly accessible with a discrete URL. Here, though, we look at N+1 fetches, or some extra way to inline the results or fetch-all in a single operation. (Both seem reasonable). Versus a "value" object, which is purely embedded within another object, and not directly referenceable. If you embed as a value, then I think you make sure it's not accessible, since it's more just an internal structure to the owning object. In this case, perhaps embedded value is the way to go, in which case, purely the following, where the only href/URL points to the referenced group: Here, you don't have to repeat user_id, since context is implicit and mandatory, as we're embedded within the object. But then we're back to the race condition of multiple updates to a single url to add/ remove groups. But that is orthogonol, and affects all fields on all resources. We just happen to think it might be triggered on this one with high probability. Perhaps time to think about optimistic locking, and a serial number or timestamp on all resources, to allow the server to perform reasonable optimistic locking. > We'll also make the membership list available read-only via the group: > > GET /groups/4321 > > > foo > ... > > > ... > > Though, does a group have memberships? Or does it have members? A "membership" seems to be a management/structural object from User perspective, but not necessarily from the group perspective, especially if it's just referencing it in a read-only fashion. I'd suggest: foo .. In this case, we're reducing the number of fetches to obtain a list of all members of a group. Having to bounce through a membership reference means 2N+1, where at least this way it's only N+1. Same inline/fetch-all notes from above may then apply to optimise some more. What an amazingly deep topic. :) -Bob From markmc at redhat.com Tue May 4 20:33:51 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Tue, 04 May 2010 21:33:51 +0100 Subject: [rest-practices] Modelling membership In-Reply-To: <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> Message-ID: <1273005231.10775.67.camel@blaa> On Tue, 2010-05-04 at 13:00 -0400, Bob McWhirter wrote: > > So, by not having it toplevel, what we're thinking is equivalent to: > > > > POST /users/1234/memberships > > > > > > > > > > Generally speaking, I'd rather see hrefs instead of IDs: > > David and Eoghan had some discussion on this on irc the other day, and we were observing: - A href is an opaque identifier returned by the server, but not guaranteed to remain the same over the long term - i.e. since the server may be upgraded and the URI structure changed or, indeed, the server might be moved - A uuid, or similar, is generated by the server, but more useful over the longer term - as long as you have the entry point URI, the UUID and the object still exists, you should be able to trivially get to it again - Most resources will also have a human readable, user-generated identifier too, but software clients should prefer the uuid So, my thinking was: - References/links in media returned by the server should use hrefs - If a client wants to retain an identifier in its DB or whatever, it should dereference the link and retain the uuid - If the client needs to supply an identifier in some media it is sending to the server, it should be able to use either a href or a uuid (maybe the human readable id too?) > I don't like this partial in-lined description of a > resource, though, as then I have to keep up with user_id separately, > it's not a part of the representation when fetched this way. Note, what you've quoted above is what is POSTed by the client, whereas the full description returned by the server does include a reference to the user e.g.: of course, that could be done with Atom links instead: I like the consistency of that, but it starts to feel like overkill pretty quickly > It's > easier to code if I know I can resolve a URL to fetch a complete > object, instead of having to considering it within some context. > Particularly if we're ignoring URL structure and being HATEOASishy. > > Depending on how "pure" you wanted to be, I might also do > > > > > > > And treat it as a collection of references. The same could be true of any collection, so I'd follow whatever pattern we were using for other collections - which generally seems to be to include the full representation rather than just a reference > This is what I mean by a > "top-level" object. One that is directly accessible with a discrete > URL. Here, though, we look at N+1 fetches, or some extra way to > inline the results or fetch-all in a single operation. (Both seem > reasonable). > > Versus a "value" object, which is purely embedded within another > object, and not directly referenceable. > > If you embed as a value, then I think you make sure it's not > accessible, since it's more just an internal structure to the owning > object. I don't see why an embedded value shouldn't also be directly accessible, it's quite common in collections AFAICS: > In this case, perhaps embedded value is the way to go, in which case, > purely the following, where the only href/URL points to the referenced > group: > > > > > > > > Here, you don't have to repeat user_id, since context is implicit and > mandatory, as we're embedded within the object. But then we're > back to the race condition of multiple updates to a single url to add/ > remove groups. Wait, why are you now dropping the idea of having membership as a distinct resource? > But that is orthogonol, and affects all fields on all > resources. We just happen to think it might be triggered on this one > with high probability. Agree on both counts > Perhaps time to think about optimistic locking, and a serial > number or timestamp on all resources, to allow the server to perform > reasonable optimistic locking. Eoghan's "reject the update if it doesn't match this ETag" idea was thinking along the same lines > > We'll also make the membership list available read-only via the group: > > > > GET /groups/4321 > > > > > > foo > > ... > > > > > > ... > > > > > > Though, does a group have memberships? Or does it have members? A > "membership" seems to be a management/structural object from User > perspective, but not necessarily from the group perspective, > especially if it's just referencing it in a read-only fashion. > > I'd suggest: > > > foo > > > > .. > > > > In this case, we're reducing the number of fetches to obtain a list of > all members of a group. Having to bounce through a membership > reference means 2N+1, where at least this way it's only N+1. Same > inline/fetch-all notes from above may then apply to optimise some more. That's all fair, except in our case: - we're talking about an 'attachment' resource, which doesn't sound as silly as a group having memberships - we have some metadata associated with the attachment resource itself, which you need some way to reference too > What an amazingly deep topic. :) I'm genuinely stunned at how REST seems like a bottomless pit of circular debates about seemingly arbitrary choices :-) Cheers, Mark. From bburke at redhat.com Tue May 4 21:24:14 2010 From: bburke at redhat.com (Bill Burke) Date: Tue, 04 May 2010 17:24:14 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> Message-ID: <4BE0907E.8060609@redhat.com> Bob McWhirter wrote: >> So, by not having it toplevel, what we're thinking is equivalent to: >> >> POST /users/1234/memberships >> >> >> >> > > Generally speaking, I'd rather see hrefs instead of IDs: > > > Not sure I agree that hrefs are better. You end up having to extract the id from the URL somehow, someway, so that you can associate the things within a relational database -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Tue May 4 21:25:42 2010 From: bburke at redhat.com (Bill Burke) Date: Tue, 04 May 2010 17:25:42 -0400 Subject: [rest-practices] [Fwd: [rhevm-api] concatenating Link headers in response to entry-point HEAD method] In-Reply-To: <1272979860.2908.103.camel@blaa> References: <1272979860.2908.103.camel@blaa> Message-ID: <4BE090D6.3080005@redhat.com> By concatenation, you mean comma delimited, right? That's what the link header spec defines. Mark McLoughlin wrote: > Hey, > > I thought this had been discussed here, but I don't think so. > > Eoghan found some issues with including multiple Link: headers. > > I've seen something similar - a ruby client was only giving me a single > Link: header. > > We've decided to concatenate our Link headers together on the server > side, as ugly as that is. > > Cheers, > Mark. > > -------- Forwarded Message -------- >> From: Eoghan Glynn >> To: rhevm-api >> Subject: [rhevm-api] concatenating Link headers in response to >> entry-point HEAD method >> Date: Mon, 26 Apr 2010 09:11:24 +0100 >> >> >> Folks, >> >> We've noticed some funnies with the handling of multiple headers of the >> same name by various REST providers, with RESTeasy concatenating the >> header values into a comma separated list, and CXF JAX-RS seemingly >> dropping all but the last header instance. >> >> In the light of the following chapter & verse from the spec: >> >> "Multiple message-header fields with the same field-name MAY be present >> in a message if and only if the entire field-value for that header field >> is defined as a comma-separated list [i.e., #(values)]. It MUST be >> possible to combine the multiple header fields into one "field-name: >> field-value" pair, without changing the semantics of the message, by >> appending each subsequent field-value to the first, each separated by a >> comma. The order in which header fields with the same field-name are >> received is therefore significant to the interpretation of the combined >> field value, and thus a proxy MUST NOT change the order of these field >> values when a message is forwarded."[1] >> >> It seems like a concatenation is the canonical form and least likely >> cause trouble along the way. So how about we simply squash together the >> Links set in the implementations of ApiResource.head()? >> >> Cheers, >> Eoghan >> >> >> [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 >> >> >> >> _______________________________________________ >> rhevm-api mailing list >> rhevm-api at lists.fedorahosted.org >> https://fedorahosted.org/mailman/listinfo/rhevm-api > > > _______________________________________________ > rest-practices mailing list > rest-practices at redhat.com > https://www.redhat.com/mailman/listinfo/rest-practices -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bmcwhirt at redhat.com Tue May 4 21:30:24 2010 From: bmcwhirt at redhat.com (Bob McWhirter) Date: Tue, 4 May 2010 17:30:24 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <4BE0907E.8060609@redhat.com> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> <4BE0907E.8060609@redhat.com> Message-ID: <7D16A6CE-3220-4715-8D25-E3CAA16E7844@redhat.com> I guess I see the need for both. I consider an href to be like a C++ pointer to *something. While that thing can certainly (and maybe absolutely should) have some other identifier. I don't mean to claim that the href is the identifier. It should be opaque, potentially ugly, but easily navigable. ID is just another property, perhaps with uniqueness constraints. -Bob On May 4, 2010, at 5:24 PM, Bill Burke wrote: > > > Bob McWhirter wrote: >>> So, by not having it toplevel, what we're thinking is equivalent to: >>> >>> POST /users/1234/memberships >>> >>> >>> >>> >> Generally speaking, I'd rather see hrefs instead of IDs: >> > > Not sure I agree that hrefs are better. You end up having to > extract the id from the URL somehow, someway, so that you can > associate the things within a relational database > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com From markmc at redhat.com Wed May 5 07:25:46 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 05 May 2010 08:25:46 +0100 Subject: [rest-practices] [Fwd: [rhevm-api] concatenating Link headers in response to entry-point HEAD method] In-Reply-To: <4BE090D6.3080005@redhat.com> References: <1272979860.2908.103.camel@blaa> <4BE090D6.3080005@redhat.com> Message-ID: <1273044346.2998.1.camel@blaa> On Tue, 2010-05-04 at 17:25 -0400, Bill Burke wrote: > By concatenation, you mean comma delimited, right? That's what the link > header spec defines. Yes, exactly. RESTeasy doesn't concatenate them by default, so we decided to do it manually to avoid any problems with clients Cheers, Mark. From bkearney at redhat.com Wed May 5 11:43:05 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 05 May 2010 07:43:05 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <7D16A6CE-3220-4715-8D25-E3CAA16E7844@redhat.com> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> <4BE0907E.8060609@redhat.com> <7D16A6CE-3220-4715-8D25-E3CAA16E7844@redhat.com> Message-ID: <4BE159C9.1040009@redhat.com> On 05/04/2010 05:30 PM, Bob McWhirter wrote: > I guess I see the need for both. I consider an href to be like a C++ > pointer to *something. While that thing can certainly (and maybe > absolutely should) have some other identifier. > > I don't mean to claim that the href is the identifier. It should be > opaque, potentially ugly, but easily navigable. ID is just another > property, perhaps with uniqueness constraints. > > -Bob I liked what lutter said earlier, which was give dserializers a way to know that they need to go get the further data. Does the existence of give you that? -- bk From bburke at redhat.com Wed May 5 12:41:32 2010 From: bburke at redhat.com (Bill Burke) Date: Wed, 05 May 2010 08:41:32 -0400 Subject: [rest-practices] Modelling membership In-Reply-To: <4BE159C9.1040009@redhat.com> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> <4BE0907E.8060609@redhat.com> <7D16A6CE-3220-4715-8D25-E3CAA16E7844@redhat.com> <4BE159C9.1040009@redhat.com> Message-ID: <4BE1677C.8000200@redhat.com> Bryan Kearney wrote: > On 05/04/2010 05:30 PM, Bob McWhirter wrote: >> I guess I see the need for both. I consider an href to be like a C++ >> pointer to *something. While that thing can certainly (and maybe >> absolutely should) have some other identifier. >> >> I don't mean to claim that the href is the identifier. It should be >> opaque, potentially ugly, but easily navigable. ID is just another >> property, perhaps with uniqueness constraints. >> >> -Bob > > I liked what lutter said earlier, which was give dserializers a way to > know that they need to go get the further data. Does the existence of > > > > give you that? > Do you gain anything by using href as the association? If you can't think of anything concrete or real, then my vote is always for simplicity of implementation. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Wed May 5 12:42:30 2010 From: bburke at redhat.com (Bill Burke) Date: Wed, 05 May 2010 08:42:30 -0400 Subject: [rest-practices] [Fwd: [rhevm-api] concatenating Link headers in response to entry-point HEAD method] In-Reply-To: <1273044346.2998.1.camel@blaa> References: <1272979860.2908.103.camel@blaa> <4BE090D6.3080005@redhat.com> <1273044346.2998.1.camel@blaa> Message-ID: <4BE167B6.5010900@redhat.com> Hmmm, i thought that apache client or the servlet container would do it automatically. Ok, i'll log this as a bug. Mark McLoughlin wrote: > On Tue, 2010-05-04 at 17:25 -0400, Bill Burke wrote: >> By concatenation, you mean comma delimited, right? That's what the link >> header spec defines. > > Yes, exactly. > > RESTeasy doesn't concatenate them by default, so we decided to do it > manually to avoid any problems with clients > > Cheers, > Mark. > -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bmcwhirt at redhat.com Wed May 5 12:45:14 2010 From: bmcwhirt at redhat.com (Bob McWhirter) Date: Wed, 5 May 2010 08:45:14 -0400 Subject: [rest-practices] [Fwd: [rhevm-api] concatenating Link headers in response to entry-point HEAD method] In-Reply-To: <4BE167B6.5010900@redhat.com> References: <1272979860.2908.103.camel@blaa> <4BE090D6.3080005@redhat.com> <1273044346.2998.1.camel@blaa> <4BE167B6.5010900@redhat.com> Message-ID: <60B1EE7E-1DE4-4A86-BEA0-E0BF141C58FE@redhat.com> I think there's a getHeader() in the servlet API that returns only the first, but a pluralized getHeaders() or similar that returns an array of all. But this is request headers, not response header processing on the client end. No idea about apache clients. -Bob On May 5, 2010, at 8:42 AM, Bill Burke wrote: > Hmmm, i thought that apache client or the servlet container would do > it automatically. Ok, i'll log this as a bug. > > Mark McLoughlin wrote: >> On Tue, 2010-05-04 at 17:25 -0400, Bill Burke wrote: >>> By concatenation, you mean comma delimited, right? That's what >>> the link header spec defines. >> Yes, exactly. >> RESTeasy doesn't concatenate them by default, so we decided to do it >> manually to avoid any problems with clients >> Cheers, >> Mark. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > _______________________________________________ > rest-practices mailing list > rest-practices at redhat.com > https://www.redhat.com/mailman/listinfo/rest-practices From markmc at redhat.com Wed May 5 20:38:41 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 05 May 2010 21:38:41 +0100 Subject: [rest-practices] Identifiers, Links and Link headers [was Re: Modelling membership] In-Reply-To: <1273005231.10775.67.camel@blaa> References: <1272979142.2908.99.camel@blaa> <1272982539.2908.111.camel@blaa> <4BE0497F.4020103@redhat.com> <1272990595.2908.164.camel@blaa> <07404844-C0D7-4ED8-B8CC-B91873F5D6A1@redhat.com> <1273005231.10775.67.camel@blaa> Message-ID: <1273091921.5618.9.camel@blaa> Hey, I've tried to distil some of this down into reasonable advice: https://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide#Identifiers Appreciate any feedback Thanks, Mark. From markmc at redhat.com Thu May 6 20:47:56 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 06 May 2010 21:47:56 +0100 Subject: [rest-practices] [deltacloud-devel] Towards declaring the Deltacloud API stable In-Reply-To: <1273105038.7253.53.camel@avon.watzmann.net> References: <1273105038.7253.53.camel@avon.watzmann.net> Message-ID: <1273178876.27025.25.camel@blaa> Hey, Just as context for others, on the rest-practices mailing list we've been trying to hash out some sort of consensus on a set of that deltacloud and other related APIs could follow. Here's the in-progress result of that: http://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide Cross-posting is evil, but I think it's worthwhile including rest-practices on this since so much of it is useful stuff to consider for all APIs. On Wed, 2010-05-05 at 17:17 -0700, David Lutterkort wrote: > Hi, > > I would like to get quickly to a point where we can declare the current > Deltacloud API stable, and be reasonably confident that we can evolve > the API in a way that will maintain backwards compatibility. > > Unfortunately, there are a few goofups in the current Deltacloud API > that we need to fix before we can declare the API stable, since fixing > them will break API compatibility. In no specific order, the issues I am > aware of are > > > * returning full objects when listing a collection: this is a > pretty serious issue. For example, for Terremark, returning a > list of full image details requires first calling to Terremark > to get a list of images without details, then retrieving the > detail of each image. That can be a long and slow process, > leading to timeout issues for clients. The best way to fix this > is to return full image details on GET /api/images from drivers > where that is cheap, and only an abbreviated list of images, > i.e. with only a URL and id for the image, from drivers like > Terremark. We'll probably have to deal with other lists in a > similar manner. Very interesting ... and your solution certainly sounds reasonable. Related, with the RHEV-M API we're experimenting with making both the id and href attributes, which would work out nice here: GET /api/images HTTP/1.1 ... rather than: GET /api/images HTTP/1.1 1234 9876 ... > * returning the representation of a new instance from a > POST /api/instances: again, in some drivers, like Terremark, > that requires making two calls to the backend cloud, which is > too fragile for this operation. Instead, the result from > POST /api/instances should contain the URL of the new instance > in a Location: header, and optionally the instance > representation for drivers where that is cheap to do. Reasonable - the instance representation in the response could be considered a convenience thing only. > * the list of actions for an instance is misleading: we report > - that is not enough, since it > gives no indication what HTTP method should be used to perform > the action. To some extent, I think it should be implicit which method should be used - and, in this case, we're using POST because we're creating a new "action instance". Then again, though, HTML forms specify whether to GET or POST ... > It's generally POST, but for destroying an instance, > you are supposed to do a DELETE on the instance URL. We > therefore need to include a method in the tag I don't follow you there ... where will you add method="DELETE"? do you mean: ... If so, I'd again say the method is implicit ... that's what DELETE means for all resources ... and you can document it pretty plainly e.g. https://fedorahosted.org/rhevm-api/wiki/CommonIdiomsForResources > * inconsistent indication of relations to other objects, e.g. the > image and the hardware profile for an instance. For the image, > we just include a tag, whereas for hardware > profile we give the URL for hte HWP and its id. There's actually > some discussion on rest-practices[1] pertaining to that, though > we haven't reached any conclusion. My vote is that the server should return in its representations, but if the client is supplying a reference it should only be required to supply > * inconsistent usage of '-' and '_' in XML tag names (e.g., > vs. ) This one is mostly cosmetic, > but if we are breaking API, we might as well fix this one, too. We went for under_score in rhevm-api Cheers, Mark. > I am sure there are other issues, and I'd appreciate if people could > help collect them, so we can address them with one last big API breakage > before we declare the API stable going forward. > > David > > [1] https://www.redhat.com/archives/rest-practices/2010-April/msg00024.html From lutter at redhat.com Thu May 6 23:40:00 2010 From: lutter at redhat.com (David Lutterkort) Date: Thu, 06 May 2010 16:40:00 -0700 Subject: [rest-practices] Variable responses Message-ID: <1273189200.7253.161.camel@avon.watzmann.net> This might be an issue that is specific to Deltacloud Core; it's definitely caused by two aspects of Deltacloud that make it different from most other REST services: * Deltacloud Core is stateless (on the server side) * Deltacloud Core is an adapter around other API's The specific issue is that for some operations, we'd like to return responses with varying levels of details. Two specific issues: Listing all images ------------------ When listing all images available in a cloud (GET /api/images), we currently return a list that contains full details for each image. That's not a workable approach for some of the backend clouds we talk to, since it forces the Deltacloud server to make n+1 requests to the backend cloud (one for the abridged list of images, n more to get details about each image) - for some backend clouds (e.g., Terremark) that becomes unacceptably slow, to the point where we need to worry about clients timing out. At the same time, we'd still like to return full details about images when that is possible cheaply (e.g., when Deltacloud talks to EC2) to save clients from having to make n+1 calls to get full image details. What I would like to do is make the result for GET /api/images vary, so that for some clouds, you'd get .. full details for the image .. ... more images ... and for others you'd only get ... ... more image references ... Creating a new instance ----------------------- In a similar vein, right now, when creating a new instance with a POST /api/instances, we return the full XML representation of the new instance. For some clouds (again, Terremark is the posterboy), this actually requires two calls inside the Deltacloud driver: one to create the instance, and one to retrieve the details of that new instance. Besides being a little slow, this is also very fragile, in that the instance creation might succeed, but retrieving details might fail, making it, at the very least, hard to indicate to the client what just happened. For that case, too, I'd like to change the API definition so that you will always get the URL for the instance details in a Location: header or abridged XML in the body, and full details only when they can be provided cheaply. My question to the list: does anybody have experience with doing this kind of variable responses ? David From lutter at redhat.com Thu May 6 23:46:38 2010 From: lutter at redhat.com (David Lutterkort) Date: Thu, 06 May 2010 16:46:38 -0700 Subject: [rest-practices] Links for actions Message-ID: <1273189598.7253.168.camel@avon.watzmann.net> One more issue from Deltacloud Core: right now, when a client gets the details about an instance, we indicate the possible actions on that instance like this: The above leaves the client guessing what method to use on the actions, and contains a little trap: for almost all actions, the method to use is POST, but for destroy, the client should use the DELETE method, since they are deleting the server side instance. You could argue that this issue is caused by using URL's that do not represent resources; but then I don't know a better way to indicate actions. My favored way out of this is to add the method the client should use to the tag, e.g. Any thoughts on this ? David From lutter at redhat.com Thu May 6 23:56:51 2010 From: lutter at redhat.com (David Lutterkort) Date: Thu, 06 May 2010 16:56:51 -0700 Subject: [rest-practices] [deltacloud-devel] Towards declaring the Deltacloud API stable In-Reply-To: <1273178876.27025.25.camel@blaa> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> Message-ID: <1273190211.7253.171.camel@avon.watzmann.net> On Thu, 2010-05-06 at 21:47 +0100, Mark McLoughlin wrote: > Just as context for others, on the rest-practices mailing list we've > been trying to hash out some sort of consensus on a set of that > deltacloud and other related APIs could follow. Should have read that before sending separate mails to rest-practices explaining the issue in more details. > Cross-posting is evil So is Reply-To ;) David From eglynn at redhat.com Fri May 7 07:45:57 2010 From: eglynn at redhat.com (Eoghan Glynn) Date: Fri, 07 May 2010 08:45:57 +0100 Subject: [rest-practices] Links for actions In-Reply-To: <1273189598.7253.168.camel@avon.watzmann.net> References: <1273189598.7253.168.camel@avon.watzmann.net> Message-ID: <1273218357.21680.22.camel@quark> On Thu, 2010-05-06 at 16:46 -0700, David Lutterkort wrote: > One more issue from Deltacloud Core: right now, when a client gets the > details about an instance, we indicate the possible actions on that > instance like this: > > > > > > > > The above leaves the client guessing what method to use on the actions, > and contains a little trap: for almost all actions, the method to use is > POST, but for destroy, the client should use the DELETE method, since > they are deleting the server side instance. > > You could argue that this issue is caused by using URL's that do not > represent resources; but then I don't know a better way to indicate > actions. > > My favored way out of this is to add the method the client should use to > the tag, e.g. > > Just playing devil's advocate for a moment, but aren't we already implying quite an amount of semantic and structural information in the rel label? We're saying the action maps to some real-world operation, that we expect an entity body of a certain type, that we may return a response of another type, or indicate a error condition with certain specific status codes and maybe even a fault entity-body with further detail. But it would probably be over the top to try to encode all that information explicitly in the link element, e.g. as most of this information can be implicit, or captured in the documentation etc. If we were to encode anything explicitly, I'm thinking the expected fine-grained media type must be more useful information, as the verb is usually POST and in any case if the client gets it wrong we can prod it into recovering gracefully via a 405 "Method Not Allowed" response with an "Allow: DELETE" header for example. Cheers, Eoghan From eglynn at redhat.com Fri May 7 08:31:04 2010 From: eglynn at redhat.com (Eoghan Glynn) Date: Fri, 07 May 2010 09:31:04 +0100 Subject: [rest-practices] Variable responses In-Reply-To: <1273189200.7253.161.camel@avon.watzmann.net> References: <1273189200.7253.161.camel@avon.watzmann.net> Message-ID: <1273221064.21680.50.camel@quark> On Thu, 2010-05-06 at 16:40 -0700, David Lutterkort wrote: > > Creating a new instance > ----------------------- > > In a similar vein, right now, when creating a new instance with a > POST /api/instances, we return the full XML representation of the new > instance. For some clouds (again, Terremark is the posterboy), this > actually requires two calls inside the Deltacloud driver: one to create > the instance, and one to retrieve the details of that new instance. > Besides being a little slow, this is also very fragile, in that the > instance creation might succeed, but retrieving details might fail, > making it, at the very least, hard to indicate to the client what just > happened. > > For that case, too, I'd like to change the API definition so that you > will always get the URL for the instance details in a Location: header > or abridged XML in the body, and full details only when they can be > provided cheaply. Whoops, I didn't realise this discussion had moved to rest-practices before replying on deltacloud-devel. Here's a slightly wacky idea on fabricating a minimal representation from the returned URI: https://fedorahosted.org/pipermail/deltacloud-devel/2010-May/001125.html /Eoghan From lutter at redhat.com Fri May 7 22:13:31 2010 From: lutter at redhat.com (David Lutterkort) Date: Fri, 07 May 2010 15:13:31 -0700 Subject: [rest-practices] Links for actions In-Reply-To: <1273218357.21680.22.camel@quark> References: <1273189598.7253.168.camel@avon.watzmann.net> <1273218357.21680.22.camel@quark> Message-ID: <1273270411.15251.26.camel@avon.watzmann.net> On Fri, 2010-05-07 at 08:45 +0100, Eoghan Glynn wrote: > On Thu, 2010-05-06 at 16:46 -0700, David Lutterkort wrote: > > One more issue from Deltacloud Core: right now, when a client gets the > > details about an instance, we indicate the possible actions on that > > instance like this: > > > > > > > > > > > > > > > > The above leaves the client guessing what method to use on the actions, > > and contains a little trap: for almost all actions, the method to use is > > POST, but for destroy, the client should use the DELETE method, since > > they are deleting the server side instance. > > > > You could argue that this issue is caused by using URL's that do not > > represent resources; but then I don't know a better way to indicate > > actions. > > > > My favored way out of this is to add the method the client should use to > > the tag, e.g. > > > > > > > Just playing devil's advocate for a moment, but aren't we already > implying quite an amount of semantic and structural information in the > rel label? Yes, that's true .. and yet, the method is what tripped at least one client up. The instance actions in the deltacloud API are otherwise very uniform: they take no parameters and no request body, and return the instance. Adding a method in this specific case allows client code to remove one special case. > But it would probably be over the top to try to encode all that > information explicitly in the link element, e.g. > > type="application/vnd.redhat.rhevm-api.Action+{xml|json}" > errorStatus="400,401,403,409,412" method="POST" /> > > as most of this information can be implicit, or captured in the > documentation etc. Agreed .. encoding all of that would be overkill. The confusion with actions and links really stems from the fact that the URL's mentioned in the link are not resources in the usual REST+HTTP sense, i.e., they only respond to one specific method, and you can't even do a GET on them. > If we were to encode anything explicitly, I'm thinking the expected > fine-grained media type must be more useful information, as the verb is > usually POST and in any case if the client gets it wrong we can prod it > into recovering gracefully via a 405 "Method Not Allowed" response with > an "Allow: DELETE" header for example. I wonder how many clients will implement this kind of error recovery properly ;) David From iheim at redhat.com Wed May 12 05:40:11 2010 From: iheim at redhat.com (Itamar Heim) Date: Wed, 12 May 2010 01:40:11 -0400 (EDT) Subject: [rest-practices] Links for actions In-Reply-To: <1273270411.15251.26.camel@avon.watzmann.net> References: <1273189598.7253.168.camel@avon.watzmann.net> <1273218357.21680.22.camel@quark> <1273270411.15251.26.camel@avon.watzmann.net> Message-ID: <147e01caf195$5ab1bfe0$10153fa0$@com> > From: rest-practices-bounces at redhat.com [mailto:rest-practices- > bounces at redhat.com] On Behalf Of David Lutterkort ... > > Just playing devil's advocate for a moment, but aren't we already > > implying quite an amount of semantic and structural information in > the > > rel label? > > Yes, that's true .. and yet, the method is what tripped at least one > client up. The instance actions in the deltacloud API are otherwise > very > uniform: they take no parameters and no request body, and return the > instance. Adding a method in this specific case allows client code to > remove one special case. > > > But it would probably be over the top to try to encode all that > > information explicitly in the link element, e.g. > > > > > type="application/vnd.redhat.rhevm-api.Action+{xml|json}" > > errorStatus="400,401,403,409,412" method="POST" /> > > > > as most of this information can be implicit, or captured in the > > documentation etc. > > Agreed .. encoding all of that would be overkill. The confusion with > actions and links really stems from the fact that the URL's mentioned > in > the link are not resources in the usual REST+HTTP sense, i.e., they > only > respond to one specific method, and you can't even do a GET on them. > > > If we were to encode anything explicitly, I'm thinking the expected > > fine-grained media type must be more useful information, as the verb > is > > usually POST and in any case if the client gets it wrong we can prod > it > > into recovering gracefully via a 405 "Method Not Allowed" response > with > > an "Allow: DELETE" header for example. > > I wonder how many clients will implement this kind of error recovery > properly ;) [IH] I agree with David. Developers should be able to use the API easily (hopefully via IDE's with intellisense actually). Forcing the developer to go and read the documentation for each API (hoping the documentation would actually be up to date) is not very friendly. From iheim at redhat.com Wed May 12 05:43:21 2010 From: iheim at redhat.com (Itamar Heim) Date: Wed, 12 May 2010 01:43:21 -0400 (EDT) Subject: [rest-practices] Links for actions In-Reply-To: <1273189598.7253.168.camel@avon.watzmann.net> References: <1273189598.7253.168.camel@avon.watzmann.net> Message-ID: <148301caf195$cbf62dd0$63e28970$@com> > From: rest-practices-bounces at redhat.com [mailto:rest-practices- > bounces at redhat.com] On Behalf Of David Lutterkort ... > One more issue from Deltacloud Core: right now, when a client gets the > details about an instance, we indicate the possible actions on that > instance like this: > > > > > > [IH] just wondering, so if I ask for a list of 500 instances, I will get all actions possible on all instances 500 times as well? isn't this mixing introspection with data? Unless I'm introspecting the web service, I'm implying I actually know what the possible actions are (i.e., I already wrote the code that uses them, and needed this data only during development time, but I don't need it during runtime[1]). [1] unless permissions are implied, and for each instance I can get a different list of . From markmc at redhat.com Wed May 12 09:23:52 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 12 May 2010 10:23:52 +0100 Subject: [rest-practices] Links for actions In-Reply-To: <1273189598.7253.168.camel@avon.watzmann.net> References: <1273189598.7253.168.camel@avon.watzmann.net> Message-ID: <1273656232.3062.39.camel@blaa> On Thu, 2010-05-06 at 16:46 -0700, David Lutterkort wrote: > One more issue from Deltacloud Core: right now, when a client gets the > details about an instance, we indicate the possible actions on that > instance like this: > > > > > > > > The above leaves the client guessing what method to use on the actions, > and contains a little trap: for almost all actions, the method to use is > POST, but for destroy, the client should use the DELETE method, since > they are deleting the server side instance. > > You could argue that this issue is caused by using URL's that do not > represent resources; but then I don't know a better way to indicate > actions. The idea is that an action *is* a resource - POST is creating an instance of it > My favored way out of this is to add the method the client should use to > the tag, e.g. > > ITYM: > Any thoughts on this ? I don't think "destroy" is an action like the others To me, it's as implicit that DELETE destroys an instance as it is that POST creates an instance e.g. you're not adding: ... So, IMHO, drop the "destroy" action and have docs like: https://fedorahosted.org/rhevm-api/wiki/CommonIdiomsForResources Cheers, Mark. From markmc at redhat.com Wed May 12 09:34:17 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 12 May 2010 10:34:17 +0100 Subject: [rest-practices] Links for actions In-Reply-To: <148301caf195$cbf62dd0$63e28970$@com> References: <1273189598.7253.168.camel@avon.watzmann.net> <148301caf195$cbf62dd0$63e28970$@com> Message-ID: <1273656857.3062.50.camel@blaa> On Wed, 2010-05-12 at 01:43 -0400, Itamar Heim wrote: > > From: rest-practices-bounces at redhat.com [mailto:rest-practices- > > bounces at redhat.com] On Behalf Of David Lutterkort > ... > > One more issue from Deltacloud Core: right now, when a client gets the > > details about an instance, we indicate the possible actions on that > > instance like this: > > > > > > > > > > > > > [IH] just wondering, so if I ask for a list of 500 instances, I will get > all actions possible on all instances 500 times as well? Yes If it becomes an issue, one way around it is to make a collection just include references e.g. ... or you could include partial representations: vm1 UP vm2 UP ... vm500 DOWN > isn't this mixing introspection with data? Yes, that's a key part of REST's "hypermedia as the engine of application state" principle The hypermedia encapsulates not only the current state of the entity, but also, how its state can be changed > Unless I'm introspecting the web service, I'm > implying I actually know what the possible actions are (i.e., I already > wrote the code that uses them, and needed this data only during > development time, but I don't need it during runtime[1]). > > [1] unless permissions are implied, and for each instance I can get a > different list of . Right, exactly - the block would only contain a "reboot" action if the user had permission to reboot that instance Cheers, Mark. From markmc at redhat.com Wed May 12 09:40:12 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 12 May 2010 10:40:12 +0100 Subject: [rest-practices] Variable responses In-Reply-To: <1273189200.7253.161.camel@avon.watzmann.net> References: <1273189200.7253.161.camel@avon.watzmann.net> Message-ID: <1273657212.3062.55.camel@blaa> On Thu, 2010-05-06 at 16:40 -0700, David Lutterkort wrote: > This might be an issue that is specific to Deltacloud Core; it's > definitely caused by two aspects of Deltacloud that make it different > from most other REST services: > > * Deltacloud Core is stateless (on the server side) > * Deltacloud Core is an adapter around other API's > > The specific issue is that for some operations, we'd like to return > responses with varying levels of details. Two specific issues: > > Listing all images > ------------------ > > When listing all images available in a cloud (GET /api/images), we > currently return a list that contains full details for each image. > > That's not a workable approach for some of the backend clouds we talk > to, since it forces the Deltacloud server to make n+1 requests to the > backend cloud (one for the abridged list of images, n more to get > details about each image) - for some backend clouds (e.g., Terremark) > that becomes unacceptably slow, to the point where we need to worry > about clients timing out. > > At the same time, we'd still like to return full details about images > when that is possible cheaply (e.g., when Deltacloud talks to EC2) to > save clients from having to make n+1 calls to get full image details. > > What I would like to do is make the result for GET /api/images vary, so > that for some clouds, you'd get > > > > .. full details for the image .. > > ... more images ... > > > and for others you'd only get > > > > ... > > ... more image references ... > The general principle here is that a resource's full representation is only guaranteed to be available from the resource's URI A client should always be prepared to de-reference the resource's URI if it feels some data is missing > Creating a new instance > ----------------------- > > In a similar vein, right now, when creating a new instance with a > POST /api/instances, we return the full XML representation of the new > instance. For some clouds (again, Terremark is the posterboy), this > actually requires two calls inside the Deltacloud driver: one to create > the instance, and one to retrieve the details of that new instance. > Besides being a little slow, this is also very fragile, in that the > instance creation might succeed, but retrieving details might fail, > making it, at the very least, hard to indicate to the client what just > happened. > > For that case, too, I'd like to change the API definition so that you > will always get the URL for the instance details in a Location: header > or abridged XML in the body, and full details only when they can be > provided cheaply. As in, a POST or PUT should not be required to return a representation of the resource in the body. If it does, it's only an optimization, otherwise the client should deference the URI in the Location header I think both of those work well Cheers, Mark. From markmc at redhat.com Wed May 12 09:44:30 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 12 May 2010 10:44:30 +0100 Subject: [rest-practices] ID as an attribute [was Re: [deltacloud-devel] Towards declaring the Deltacloud API stable] In-Reply-To: <1273178876.27025.25.camel@blaa> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> Message-ID: <1273657470.3062.58.camel@blaa> On Thu, 2010-05-06 at 21:47 +0100, Mark McLoughlin wrote: > Related, with the RHEV-M API we're experimenting with making both the > id and href attributes, which would work out nice here: > > GET /api/images HTTP/1.1 > > > > > ... > > > rather than: > > GET /api/images HTTP/1.1 > > > > 1234 > > > 9876 > > ... > Just wanted to bring this up again - it got lost in the mix I don't feel very strongly about it either way, but I do prefer: POST /storagedomains/1234/attachments HTTP/1.1 over: POST /storagedomains/1234/attachments HTTP/1.1 "5678" Cheers, Mark. From iheim at redhat.com Wed May 12 10:56:29 2010 From: iheim at redhat.com (Itamar Heim) Date: Wed, 12 May 2010 06:56:29 -0400 (EDT) Subject: [rest-practices] Links for actions In-Reply-To: <1273656857.3062.50.camel@blaa> References: <1273189598.7253.168.camel@avon.watzmann.net> <148301caf195$cbf62dd0$63e28970$@com> <1273656857.3062.50.camel@blaa> Message-ID: <156101caf1c1$88c00c30$9a402490$@com> > From: Mark McLoughlin [mailto:markmc at redhat.com] ... > > > > > > > > > > > > > > > > > > > > [IH] just wondering, so if I ask for a list of 500 instances, I will > get > > all actions possible on all instances 500 times as well? > > Yes > > If it becomes an issue, one way around it is to make a collection just > include references e.g. > > > > > ... > > > > or you could include partial representations: > > > > vm1 > UP > > > vm2 > UP > > ... > > vm500 > DOWN > > [IH] how would the client pass the request or a partial entity (parameter wise)? > > > isn't this mixing introspection with data? > > Yes, that's a key part of REST's "hypermedia as the engine of > application state" principle [IH] that's quite an overhead of data when it is not needed. > > The hypermedia encapsulates not only the current state of the entity, > but also, how its state can be changed From bkearney at redhat.com Wed May 12 12:32:22 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 12 May 2010 08:32:22 -0400 Subject: [rest-practices] ID as an attribute [was Re: [deltacloud-devel] Towards declaring the Deltacloud API stable] In-Reply-To: <1273657470.3062.58.camel@blaa> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> <1273657470.3062.58.camel@blaa> Message-ID: <4BEA9FD6.3020208@redhat.com> On 05/12/2010 05:44 AM, Mark McLoughlin wrote: > On Thu, 2010-05-06 at 21:47 +0100, Mark McLoughlin wrote: > >> Related, with the RHEV-M API we're experimenting with making both the >> id and href attributes, which would work out nice here: >> >> GET /api/images HTTP/1.1 >> >> >> >> >> ... >> >> >> rather than: >> >> GET /api/images HTTP/1.1 >> >> >> >> 1234 >> >> >> 9876 >> >> ... >> > > Just wanted to bring this up again - it got lost in the mix > > I don't feel very strongly about it either way, but I do prefer: > > POST /storagedomains/1234/attachments HTTP/1.1 > > > > over: > > POST /storagedomains/1234/attachments HTTP/1.1 > > "5678" This may be a Java centric issue, but will the former result in json which looks like: {datacenter: {@id = "5678}} That is kinda ugly, which is why I have tried to use elements where possible. I do not remember if it was Jersey or RestEasy which did this. -- bk From bkearney at redhat.com Wed May 12 12:40:44 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Wed, 12 May 2010 08:40:44 -0400 Subject: [rest-practices] Variable responses In-Reply-To: <1273657212.3062.55.camel@blaa> References: <1273189200.7253.161.camel@avon.watzmann.net> <1273657212.3062.55.camel@blaa> Message-ID: <4BEAA1CC.7020507@redhat.com> On 05/12/2010 05:40 AM, Mark McLoughlin wrote: > On Thu, 2010-05-06 at 16:40 -0700, David Lutterkort wrote: >> This might be an issue that is specific to Deltacloud Core; it's >> definitely caused by two aspects of Deltacloud that make it different >> from most other REST services: >> >> * Deltacloud Core is stateless (on the server side) >> * Deltacloud Core is an adapter around other API's >> >> The specific issue is that for some operations, we'd like to return >> responses with varying levels of details. Two specific issues: >> >> Listing all images >> ------------------ >> >> When listing all images available in a cloud (GET /api/images), we >> currently return a list that contains full details for each image. >> >> That's not a workable approach for some of the backend clouds we talk >> to, since it forces the Deltacloud server to make n+1 requests to the >> backend cloud (one for the abridged list of images, n more to get >> details about each image) - for some backend clouds (e.g., Terremark) >> that becomes unacceptably slow, to the point where we need to worry >> about clients timing out. >> >> At the same time, we'd still like to return full details about images >> when that is possible cheaply (e.g., when Deltacloud talks to EC2) to >> save clients from having to make n+1 calls to get full image details. >> >> What I would like to do is make the result for GET /api/images vary, so >> that for some clouds, you'd get >> >> >> >> .. full details for the image .. >> >> ... more images ... >> >> >> and for others you'd only get >> >> >> >> ... >> >> ... more image references ... >> > > The general principle here is that a resource's full representation is > only guaranteed to be available from the resource's URI > > A client should always be prepared to de-reference the resource's URI if > it feels some data is missing > >> Creating a new instance >> ----------------------- >> >> In a similar vein, right now, when creating a new instance with a >> POST /api/instances, we return the full XML representation of the new >> instance. For some clouds (again, Terremark is the posterboy), this >> actually requires two calls inside the Deltacloud driver: one to create >> the instance, and one to retrieve the details of that new instance. >> Besides being a little slow, this is also very fragile, in that the >> instance creation might succeed, but retrieving details might fail, >> making it, at the very least, hard to indicate to the client what just >> happened. >> >> For that case, too, I'd like to change the API definition so that you >> will always get the URL for the instance details in a Location: header >> or abridged XML in the body, and full details only when they can be >> provided cheaply. > > As in, a POST or PUT should not be required to return a representation > of the resource in the body. If it does, it's only an optimization, > otherwise the client should deference the URI in the Location header > > I think both of those work well Is there a standard which frameworks expect? I think that basic save logic in activeresource would be fine with this. For that matter, do we have a list of current viable REST toolsets? It would be good to have this as means of testing how the standards work across the tool chains. An API standard is nice, having it work in the exisiting frameworks is nicer. We have been using the following for clients: Python: - Custom using Net:http Ruby: - Custom - Active Resource Java - Jersey - RestEasy -- bk From markmc at redhat.com Wed May 12 17:33:24 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 12 May 2010 18:33:24 +0100 Subject: [rest-practices] Links for actions In-Reply-To: <156101caf1c1$88c00c30$9a402490$@com> References: <1273189598.7253.168.camel@avon.watzmann.net> <148301caf195$cbf62dd0$63e28970$@com> <1273656857.3062.50.camel@blaa> <156101caf1c1$88c00c30$9a402490$@com> Message-ID: <1273685604.3062.111.camel@blaa> On Wed, 2010-05-12 at 06:56 -0400, Itamar Heim wrote: > > From: Mark McLoughlin [mailto:markmc at redhat.com] > > > > vm500 > > DOWN > > > > > [IH] how would the client pass the request or a partial entity (parameter > wise)? you mean something this? https://fedorahosted.org/rhevm-api/wiki/StorageDomainsCollection POST /storagedomains/{id}/initialize HTTP/1.1 Content-type: application/xml Content-length: {length} i.e. the client doesn't have to pass an complete representation of the host > > > isn't this mixing introspection with data? > > > > Yes, that's a key part of REST's "hypermedia as the engine of > > application state" principle > [IH] that's quite an overhead of data when it is not needed. You gave a good example of why its needed - it lets the client know up front which operations are applicable for a given resource, without the client having to make extra requests to query it Cheers, Mark. From markmc at redhat.com Wed May 12 17:40:16 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 12 May 2010 18:40:16 +0100 Subject: [rest-practices] Variable responses In-Reply-To: <4BEAA1CC.7020507@redhat.com> References: <1273189200.7253.161.camel@avon.watzmann.net> <1273657212.3062.55.camel@blaa> <4BEAA1CC.7020507@redhat.com> Message-ID: <1273686016.3062.118.camel@blaa> On Wed, 2010-05-12 at 08:40 -0400, Bryan Kearney wrote: > For that matter, do we have a list of current viable REST toolsets? It > would be good to have this as means of testing how the standards work > across the tool chains. An API standard is nice, having it work in the > exisiting frameworks is nicer. Completely agree. That's kind of where I was going with: http://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide#Language.2FPlatform_Considerations > We have been using the following for clients: > > Python: > - Custom using Net:http > > Ruby: > - Custom > - Active Resource > > Java > - Jersey > - RestEasy Cool, as always feel free to add to the wiki Cheers, Mark. From iheim at redhat.com Wed May 12 17:40:58 2010 From: iheim at redhat.com (Itamar Heim) Date: Wed, 12 May 2010 13:40:58 -0400 (EDT) Subject: [rest-practices] Links for actions In-Reply-To: <1273685604.3062.111.camel@blaa> References: <1273189598.7253.168.camel@avon.watzmann.net> <148301caf195$cbf62dd0$63e28970$@com> <1273656857.3062.50.camel@blaa> <156101caf1c1$88c00c30$9a402490$@com> <1273685604.3062.111.camel@blaa> Message-ID: <010901caf1fa$87150460$953f0d20$@com> > From: Mark McLoughlin [mailto:markmc at redhat.com] ... > On Wed, 2010-05-12 at 06:56 -0400, Itamar Heim wrote: > > > From: Mark McLoughlin [mailto:markmc at redhat.com] > > > > > > > vm500 > > > DOWN > > > > > > > > [IH] how would the client pass the request or a partial entity > (parameter > > wise)? > > you mean something this? > > https://fedorahosted.org/rhevm-api/wiki/StorageDomainsCollection > > POST /storagedomains/{id}/initialize HTTP/1.1 > Content-type: application/xml > Content-length: {length} > > > > > > i.e. the client doesn't have to pass an complete representation of the > host [IH] I meant when client wants to get back a list of all storage domains, but only wants their ID and Name, and not all their statistics for example. > > > > > isn't this mixing introspection with data? > > > > > > Yes, that's a key part of REST's "hypermedia as the engine of > > > application state" principle > > [IH] that's quite an overhead of data when it is not needed. > > You gave a good example of why its needed - it lets the client know up > front which operations are applicable for a given resource, without the > client having to make extra requests to query it [IH] I gave an example of when I may want it, but there are still plenty of use cases where all objects may have same operations on them and I wouldn't want this overhead. From lutter at redhat.com Wed May 12 23:35:32 2010 From: lutter at redhat.com (David Lutterkort) Date: Wed, 12 May 2010 16:35:32 -0700 Subject: [rest-practices] ID as an attribute [was Re: [deltacloud-devel] Towards declaring the Deltacloud API stable] In-Reply-To: <1273657470.3062.58.camel@blaa> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> <1273657470.3062.58.camel@blaa> Message-ID: <1273707332.15251.317.camel@avon.watzmann.net> On Wed, 2010-05-12 at 10:44 +0100, Mark McLoughlin wrote: > On Thu, 2010-05-06 at 21:47 +0100, Mark McLoughlin wrote: > > > Related, with the RHEV-M API we're experimenting with making both the > > id and href attributes, which would work out nice here: > > > > GET /api/images HTTP/1.1 > > > > > > > > > > ... > > > > > > rather than: > > > > GET /api/images HTTP/1.1 > > > > > > > > 1234 > > > > > > 9876 > > > > ... > > > > Just wanted to bring this up again - it got lost in the mix > > I don't feel very strongly about it either way, but I do prefer: > > POST /storagedomains/1234/attachments HTTP/1.1 > > > > over: > > POST /storagedomains/1234/attachments HTTP/1.1 > > "5678" /me too. I have some hesitation about an 'id' attribute, simply because it has a very special meaning in HTML (and some XML schemata); I was thinking of using the following scheme for references in the Deltacloud API: corresponding to the three types of identifiers listed in [1], where href has a URL, key has some sort of primary key (often the last part of the URL, but clients don't need to care about that), and label the human-readable identifier. For most Deltacloud objects, there will actually not be a label. David [1] http://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide#Identifiers David From jbowes at redhat.com Fri May 21 14:46:23 2010 From: jbowes at redhat.com (James Bowes) Date: Fri, 21 May 2010 10:46:23 -0400 Subject: [rest-practices] JSON style guidelines? Message-ID: <20100521144623.GB19381@tiny.local> Hi all: For Candlepin, our primary media type is JSON. We've been using RESTEasy with Jettison, which does: JAXB annotated object -> XML -> JSON As such, we're really representing the XML mappings of our objects in JSON, rather than the objects themselves. Jettison is doing this with the BadgerFish mapping convention, so we get JSON like: { "consumer" : { "name" : "my consumer", "facts" : { "entry" : [ { "key" : "memory", "value" : 12345 }, { "key" : "memory.free", "value" : 60 } ] } } } My feeling is that if you're using JSON, you want it to be as compact as possible. To that end, I've been working on using Jackson instead, which cuts out the XML middleman, giving us JSON like: { "name" : "my consumer", "facts" : { "memory" : 12345, "memory.free" : 60 } } The mapping of the "facts" hash is much nicer, but you do lose that beginning "consumer" type hint. I was thinking we could add a "type_hint" field inside the returned object instead, or just rely on the client to know what type to expect based on the url they hit. What is the opinion of the list? Is anyone aware of any gotchas with the various JSON styles for autogenerating class representations? Thanks, -James -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From bburke at redhat.com Fri May 21 15:35:45 2010 From: bburke at redhat.com (Bill Burke) Date: Fri, 21 May 2010 11:35:45 -0400 Subject: [rest-practices] Links for actions In-Reply-To: <1273270411.15251.26.camel@avon.watzmann.net> References: <1273189598.7253.168.camel@avon.watzmann.net> <1273218357.21680.22.camel@quark> <1273270411.15251.26.camel@avon.watzmann.net> Message-ID: <4BF6A851.3060005@redhat.com> David Lutterkort wrote: > On Fri, 2010-05-07 at 08:45 +0100, Eoghan Glynn wrote: >> On Thu, 2010-05-06 at 16:46 -0700, David Lutterkort wrote: >>> One more issue from Deltacloud Core: right now, when a client gets the >>> details about an instance, we indicate the possible actions on that >>> instance like this: >>> >>> >>> >>> >>> >>> >>> >>> The above leaves the client guessing what method to use on the actions, >>> and contains a little trap: for almost all actions, the method to use is >>> POST, but for destroy, the client should use the DELETE method, since >>> they are deleting the server side instance. >>> >>> You could argue that this issue is caused by using URL's that do not >>> represent resources; but then I don't know a better way to indicate >>> actions. >>> >>> My favored way out of this is to add the method the client should use to >>> the tag, e.g. >>> >>> >> >> Just playing devil's advocate for a moment, but aren't we already >> implying quite an amount of semantic and structural information in the >> rel label? > > Yes, that's true .. and yet, the method is what tripped at least one > client up. The instance actions in the deltacloud API are otherwise very > uniform: they take no parameters and no request body, and return the > instance. Adding a method in this specific case allows client code to > remove one special case. > >> But it would probably be over the top to try to encode all that >> information explicitly in the link element, e.g. >> >> > type="application/vnd.redhat.rhevm-api.Action+{xml|json}" >> errorStatus="400,401,403,409,412" method="POST" /> >> >> as most of this information can be implicit, or captured in the >> documentation etc. > > Agreed .. encoding all of that would be overkill. The confusion with > actions and links really stems from the fact that the URL's mentioned in > the link are not resources in the usual REST+HTTP sense, i.e., they only > respond to one specific method, and you can't even do a GET on them. > +1. There should be no special meaning assigned to error codes. The client should already be aware, through the HTTP specification, what error codes it needs to handle and how. If you need more fine grain error messages, then IMO, enclose a media type that tells the client what to do. What I've encountered so far, is that if you rely on the HTTP specification, things get simpler and change is easier to introduce. The key always is to have the client react to what the server is telling it what to do. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bburke at redhat.com Fri May 21 15:44:16 2010 From: bburke at redhat.com (Bill Burke) Date: Fri, 21 May 2010 11:44:16 -0400 Subject: [rest-practices] JSON style guidelines? In-Reply-To: <20100521144623.GB19381@tiny.local> References: <20100521144623.GB19381@tiny.local> Message-ID: <4BF6AA50.4000608@redhat.com> +1 on using Jackson. I'm not impressed with Jettison's stability and Jackson seems to have a lot more momentum behind it. Jackson also I think supports JAXB annotations. The Jettison approach + BadgerFish is only interesting, IMO, if you want to have an XML Schema that defines both your XML and JSON documents that represent the same information. There's also JSON schema which I believe Jackson also supports. For all, i think you should rely on the client to know what type to expect based on the URL they client *AND* the Content-Type returned. i.e. application/customer+json *+json isn't "standardized", but IMO, that *+type style should be well understood. James Bowes wrote: > Hi all: > > For Candlepin, our primary media type is JSON. We've been using > RESTEasy with Jettison, which does: > JAXB annotated object -> XML -> JSON > As such, we're really representing the XML mappings of our objects in > JSON, rather than the objects themselves. Jettison is doing this with > the BadgerFish mapping convention, so we get JSON like: > { > "consumer" : { > "name" : "my consumer", > "facts" : { > "entry" : [ > { "key" : "memory", "value" : 12345 }, > { "key" : "memory.free", "value" : 60 } > ] > } > } > } > > My feeling is that if you're using JSON, you want it to be as compact as > possible. To that end, I've been working on using Jackson instead, which > cuts out the XML middleman, giving us JSON like: > { > "name" : "my consumer", > "facts" : { > "memory" : 12345, > "memory.free" : 60 > } > } > > The mapping of the "facts" hash is much nicer, but you do lose that > beginning "consumer" type hint. I was thinking we could add a > "type_hint" field inside the returned object instead, or just rely on > the client to know what type to expect based on the url they hit. > > What is the opinion of the list? Is anyone aware of any gotchas with the > various JSON styles for autogenerating class representations? > > Thanks, > > -James > > > ------------------------------------------------------------------------ > > _______________________________________________ > rest-practices mailing list > rest-practices at redhat.com > https://www.redhat.com/mailman/listinfo/rest-practices -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From eglynn at redhat.com Mon May 24 20:10:51 2010 From: eglynn at redhat.com (Eoghan Glynn) Date: Mon, 24 May 2010 21:10:51 +0100 Subject: [rest-practices] Use of URI templates for query representation Message-ID: <1274731851.9224.126.camel@quark> Hi Folks, We've been discussing query representations over on the rhevm-api list[1], coming to the conclusion that a query parameter is the cleanest way of capturing the optionality of a query constraint on a collection- oriented GET. The key point is to avoid leaking detailed knowledge of the URI structure onto the client side, by handing out the URI templates via link headers and/or inline link elements, so that a well-behaved client can treat these as opaque strings. However this opacity is a bit limited in the more widely supported version of the URI template draft, as the client still needs to know the names of both the query parameter and the substitution variable (i.e. "search" and "query" in http://rhevm/vms?search={query}). Interestingly, the latest draft[2] allows a more compact syntax: http://rhevm/vms{?search} further reducing the client's exposure to the URI structure, by eliminating the named substitution variable. Unfortunately though it's not a reasonable proposition for us to use this latest syntax, as client-side support still seems to be fairly lacking. In fact even the "old" template syntax can be frustrated by the client-side API, for example the JAX-RS UriBuilder doesn't provide a straightforward way of treating the URI template as an opaque string, as it forces the client to parse out the query parameter and substitution variable. For Java clients, the Abdera URI template support seems a superior choice in this regard. In any case if anyone has any strong feelings on the use of URI templates, please shout out. Cheers, Eoghan [1] https://fedorahosted.org/pipermail/rhevm-api/2010-May/000213.html [2] http://tools.ietf.org/html/draft-gregorio-uritemplate-04 From bkearney at redhat.com Tue May 25 12:30:48 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Tue, 25 May 2010 08:30:48 -0400 Subject: [rest-practices] Use of URI templates for query representation In-Reply-To: <1274731851.9224.126.camel@quark> References: <1274731851.9224.126.camel@quark> Message-ID: <4BFBC2F8.6010504@redhat.com> On 05/24/2010 04:10 PM, Eoghan Glynn wrote: > > Hi Folks, > > We've been discussing query representations over on the rhevm-api > list[1], coming to the conclusion that a query parameter is the cleanest > way of capturing the optionality of a query constraint on a collection- > oriented GET. > > The key point is to avoid leaking detailed knowledge of the URI > structure onto the client side, by handing out the URI templates via > link headers and/or inline link elements, so that a well-behaved client > can treat these as opaque strings. > > However this opacity is a bit limited in the more widely supported > version of the URI template draft, as the client still needs to know the > names of both the query parameter and the substitution variable (i.e. > "search" and "query" in http://rhevm/vms?search={query}). > > Interestingly, the latest draft[2] allows a more compact syntax: > > http://rhevm/vms{?search} > > further reducing the client's exposure to the URI structure, by > eliminating the named substitution variable. Unfortunately though it's > not a reasonable proposition for us to use this latest syntax, as > client-side support still seems to be fairly lacking. > > In fact even the "old" template syntax can be frustrated by the > client-side API, for example the JAX-RS UriBuilder doesn't provide a > straightforward way of treating the URI template as an opaque string, as > it forces the client to parse out the query parameter and substitution > variable. For Java clients, the Abdera URI template support seems a > superior choice in this regard. > > In any case if anyone has any strong feelings on the use of URI > templates, please shout out. > > Cheers, > Eoghan > How do you see this working in the client? I tend to not be opposed to hiding the url.. but I am curious how this would look. -- bk From eglynn at redhat.com Tue May 25 12:49:04 2010 From: eglynn at redhat.com (Eoghan Glynn) Date: Tue, 25 May 2010 13:49:04 +0100 Subject: [rest-practices] Use of URI templates for query representation In-Reply-To: <4BFBC2F8.6010504@redhat.com> References: <1274731851.9224.126.camel@quark> <4BFBC2F8.6010504@redhat.com> Message-ID: <1274791744.9224.179.camel@quark> On Tue, 2010-05-25 at 08:30 -0400, Bryan Kearney wrote: > > How do you see this working in the client? I tend to not be opposed to > hiding the url.. but I am curious how this would look. For example python client usage (with the googlecode uri-template library to do the parsing), see testQuery() in http://bit.ly/python-test For example Java client usage (with the Apache Abdera template support to the parsing), see getBaseUri() in http://bit.ly/cli-client In both cases, the templated URI is returned via a Link header in the response to a HEAD request on an entry-point URI. The template URI is not completely opaque to the client, as it needs to know the query parameter and substitution variable name (though the latter would be eliminated in the latest version of the URI template draft). Cheers, Eoghan From markmc at redhat.com Thu May 27 10:14:39 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 11:14:39 +0100 Subject: [rest-practices] Links for actions In-Reply-To: <1273656232.3062.39.camel@blaa> References: <1273189598.7253.168.camel@avon.watzmann.net> <1273656232.3062.39.camel@blaa> Message-ID: <1274955279.29945.90.camel@blaa> Hi David, On Wed, 2010-05-12 at 10:23 +0100, Mark McLoughlin wrote: > On Thu, 2010-05-06 at 16:46 -0700, David Lutterkort wrote: > > One more issue from Deltacloud Core: right now, when a client gets the > > details about an instance, we indicate the possible actions on that > > instance like this: > > > > > > > > > > > > > > > > The above leaves the client guessing what method to use on the actions, > > and contains a little trap: for almost all actions, the method to use is > > POST, but for destroy, the client should use the DELETE method, since > > they are deleting the server side instance. > > > > You could argue that this issue is caused by using URL's that do not > > represent resources; but then I don't know a better way to indicate > > actions. > > The idea is that an action *is* a resource - POST is creating an > instance of it > > > My favored way out of this is to add the method the client should use to > > the tag, e.g. > > > > > > ITYM: > > > > > Any thoughts on this ? > > I don't think "destroy" is an action like the others > > To me, it's as implicit that DELETE destroys an instance as it is that > POST creates an instance e.g. you're not adding: > > > > > > ... > > > So, IMHO, drop the "destroy" action and have docs like: > > https://fedorahosted.org/rhevm-api/wiki/CommonIdiomsForResources Any more thoughts? Given that the use of GET, POST and PUT is implied, I guess I just don't understand what's special about DELETE that we'd have a to describe it Btw, I've added a section to the style guide: https://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide#CRUD Cheers, Mark. From markmc at redhat.com Thu May 27 10:18:33 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 11:18:33 +0100 Subject: [rest-practices] JSON style guidelines? In-Reply-To: <20100521144623.GB19381@tiny.local> References: <20100521144623.GB19381@tiny.local> Message-ID: <1274955513.29945.94.camel@blaa> Hi James, On Fri, 2010-05-21 at 10:46 -0400, James Bowes wrote: > Hi all: > > For Candlepin, our primary media type is JSON. We've been using > RESTEasy with Jettison, which does: > JAXB annotated object -> XML -> JSON > As such, we're really representing the XML mappings of our objects in > JSON, rather than the objects themselves. Jettison is doing this with > the BadgerFish mapping convention, so we get JSON like: > { > "consumer" : { > "name" : "my consumer", > "facts" : { > "entry" : [ > { "key" : "memory", "value" : 12345 }, > { "key" : "memory.free", "value" : 60 } > ] > } > } > } > > My feeling is that if you're using JSON, you want it to be as compact as > possible. To that end, I've been working on using Jackson instead, which > cuts out the XML middleman, giving us JSON like: > { > "name" : "my consumer", > "facts" : { > "memory" : 12345, > "memory.free" : 60 > } > } In rhevm-api, we initially went with Jettison and BadgerFish, but without giving it much thought. You make a good point about compactness, I think we'll trying moving to Jackson too. > The mapping of the "facts" hash is much nicer, but you do lose that > beginning "consumer" type hint. I was thinking we could add a > "type_hint" field inside the returned object instead, or just rely on > the client to know what type to expect based on the url they hit. One place this would hurt us in rhevm-api is that our error handling will return a description, so e.g. PUT /resources/12345 HTTP/1.1 Content-Type: application/xml HTTP/1.1 409 Conflict Content-Type: application/xml Broken immutability constraint Attempt to set immutable field: id i.e. the client is expecting a returned, but we actually return a . The only way the client could predict that is to look at the status code or the root element of the document. With the JSON mapping you describe, you lose the equivalent of the root element Bill makes a valid point that if we instead used explicit media types for the different formats, things would be a lot clearer for the client > What is the opinion of the list? Is anyone aware of any gotchas with the > various JSON styles for autogenerating class representations? Btw, it'd be cool if you could roughly document the mapping here: http://fedoraproject.org/wiki/Cloud_APIs_REST_Style_Guide#JSON Would also be good to know what mapping JSON deltacloud is using? Cheers, Mark. From markmc at redhat.com Thu May 27 10:29:16 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 11:29:16 +0100 Subject: [rest-practices] ID as an attribute [was Re: [deltacloud-devel] Towards declaring the Deltacloud API stable] In-Reply-To: <4BEA9FD6.3020208@redhat.com> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> <1273657470.3062.58.camel@blaa> <4BEA9FD6.3020208@redhat.com> Message-ID: <1274956156.29945.97.camel@blaa> On Wed, 2010-05-12 at 08:32 -0400, Bryan Kearney wrote: > On 05/12/2010 05:44 AM, Mark McLoughlin wrote: > > On Thu, 2010-05-06 at 21:47 +0100, Mark McLoughlin wrote: > > > >> Related, with the RHEV-M API we're experimenting with making both the > >> id and href attributes, which would work out nice here: > >> > >> GET /api/images HTTP/1.1 > >> > >> > >> > >> > >> ... > >> > >> > >> rather than: > >> > >> GET /api/images HTTP/1.1 > >> > >> > >> > >> 1234 > >> > >> > >> 9876 > >> > >> ... > >> > > > > Just wanted to bring this up again - it got lost in the mix > > > > I don't feel very strongly about it either way, but I do prefer: > > > > POST /storagedomains/1234/attachments HTTP/1.1 > > > > > > > > over: > > > > POST /storagedomains/1234/attachments HTTP/1.1 > > > > "5678" > > > This may be a Java centric issue, but will the former result in json > which looks like: > > {datacenter: {@id = "5678}} > > That is kinda ugly, which is why I have tried to use elements where > possible. I do not remember if it was Jersey or RestEasy which did this. It'd be nice if we avoid letting JSON mapping issues dictate the XML design (and vice versa) ... Is there any way to easily customize the JSON mapping so that these attributes can be serialized like other properties? Cheers, Mark. From markmc at redhat.com Thu May 27 10:49:11 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 11:49:11 +0100 Subject: [rest-practices] Relative uris Message-ID: <1274957351.29945.111.camel@blaa> Hi, For rhevm-api, we're thinking of changing all the links within our documents to be relative to the entry point URI So, e.g. HEAD /api/ HTTP/1.1 Host: foobar HTTP/1.1 200 OK Link: ... Rather than including a element to describe how relative URIs should be resolved, we're thinking of just saying that the base is defined to be the entry point URI We don't think this imposes a big burden on clients, but allows the resource representations to be a good bit more compact and also means that URIs within the documents don't need to be generated in a request specific context However, for link headers, we're not so keen on going with relative URIs Any thoughts? Thanks, Mark. From markmc at redhat.com Thu May 27 10:56:40 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 11:56:40 +0100 Subject: [rest-practices] Use of URI templates for query representation In-Reply-To: <4BFBC2F8.6010504@redhat.com> References: <1274731851.9224.126.camel@quark> <4BFBC2F8.6010504@redhat.com> Message-ID: <1274957800.29945.117.camel@blaa> On Tue, 2010-05-25 at 08:30 -0400, Bryan Kearney wrote: > On 05/24/2010 04:10 PM, Eoghan Glynn wrote: > > > > Hi Folks, > > > > We've been discussing query representations over on the rhevm-api > > list[1], coming to the conclusion that a query parameter is the cleanest > > way of capturing the optionality of a query constraint on a collection- > > oriented GET. > > > > The key point is to avoid leaking detailed knowledge of the URI > > structure onto the client side, by handing out the URI templates via > > link headers and/or inline link elements, so that a well-behaved client > > can treat these as opaque strings. > > > > However this opacity is a bit limited in the more widely supported > > version of the URI template draft, as the client still needs to know the > > names of both the query parameter and the substitution variable (i.e. > > "search" and "query" in http://rhevm/vms?search={query}). > > > > Interestingly, the latest draft[2] allows a more compact syntax: > > > > http://rhevm/vms{?search} > > > > further reducing the client's exposure to the URI structure, by > > eliminating the named substitution variable. Unfortunately though it's > > not a reasonable proposition for us to use this latest syntax, as > > client-side support still seems to be fairly lacking. > > > > In fact even the "old" template syntax can be frustrated by the > > client-side API, for example the JAX-RS UriBuilder doesn't provide a > > straightforward way of treating the URI template as an opaque string, as > > it forces the client to parse out the query parameter and substitution > > variable. For Java clients, the Abdera URI template support seems a > > superior choice in this regard. > > > > In any case if anyone has any strong feelings on the use of URI > > templates, please shout out. > > > > Cheers, > > Eoghan > > > > > How do you see this working in the client? I tend to not be opposed to > hiding the url.. but I am curious how this would look. One idea we've talked about is to put a limit in advance on the subset of the URI template spec the server will ever make use of. This would greatly reduce the complexity on the client side Cheers, Mark. From bkearney at redhat.com Thu May 27 11:47:35 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 27 May 2010 07:47:35 -0400 Subject: [rest-practices] Relative uris In-Reply-To: <1274957351.29945.111.camel@blaa> References: <1274957351.29945.111.camel@blaa> Message-ID: <4BFE5BD7.7020201@redhat.com> On 05/27/2010 06:49 AM, Mark McLoughlin wrote: > Hi, > > For rhevm-api, we're thinking of changing all the links within our > documents to be relative to the entry point URI > > So, e.g. > > HEAD /api/ HTTP/1.1 > Host: foobar > > HTTP/1.1 200 OK > Link: > > GET /api/resources/ HTTP/1.1 > Host: foobar > > HTTP/1.1 200 OK > Content-Type: application/xml > > > > ... > > > > Rather than including a element to describe how relative URIs > should be resolved, we're thinking of just saying that the base is > defined to be the entry point URI > > We don't think this imposes a big burden on clients, but allows the > resource representations to be a good bit more compact and also means > that URIs within the documents don't need to be generated in a request > specific context > > However, for link headers, we're not so keen on going with relative URIs > > Any thoughts? > > Thanks, > Mark. > What happens if you do JSON over a message bus? Are you using the same serialization? That may be an edge case, but I would not want ot have to manage multiple serialiing layers per transprot. -- bk From bkearney at redhat.com Thu May 27 11:56:01 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 27 May 2010 07:56:01 -0400 Subject: [rest-practices] ID as an attribute [was Re: [deltacloud-devel] Towards declaring the Deltacloud API stable] In-Reply-To: <1274956156.29945.97.camel@blaa> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> <1273657470.3062.58.camel@blaa> <4BEA9FD6.3020208@redhat.com> <1274956156.29945.97.camel@blaa> Message-ID: <4BFE5DD1.3070308@redhat.com> On 05/27/2010 06:29 AM, Mark McLoughlin wrote: > On Wed, 2010-05-12 at 08:32 -0400, Bryan Kearney wrote: >> On 05/12/2010 05:44 AM, Mark McLoughlin wrote: >>> On Thu, 2010-05-06 at 21:47 +0100, Mark McLoughlin wrote: >>> >>>> Related, with the RHEV-M API we're experimenting with making both the >>>> id and href attributes, which would work out nice here: >>>> >>>> GET /api/images HTTP/1.1 >>>> >>>> >>>> >>>> >>>> ... >>>> >>>> >>>> rather than: >>>> >>>> GET /api/images HTTP/1.1 >>>> >>>> >>>> >>>> 1234 >>>> >>>> >>>> 9876 >>>> >>>> ... >>>> >>> >>> Just wanted to bring this up again - it got lost in the mix >>> >>> I don't feel very strongly about it either way, but I do prefer: >>> >>> POST /storagedomains/1234/attachments HTTP/1.1 >>> >>> >>> >>> over: >>> >>> POST /storagedomains/1234/attachments HTTP/1.1 >>> >>> "5678" >> >> >> This may be a Java centric issue, but will the former result in json >> which looks like: >> >> {datacenter: {@id = "5678}} >> >> That is kinda ugly, which is why I have tried to use elements where >> possible. I do not remember if it was Jersey or RestEasy which did this. > > It'd be nice if we avoid letting JSON mapping issues dictate the XML > design (and vice versa) ... > > Is there any way to easily customize the JSON mapping so that these > attributes can be serialized like other properties? > > Cheers, > Mark. > We just moved to jettiso, so I will take a look. I know on our side.. JSON is the preferred transport.. XML is the subordinate. I assume for you guys it is the other way around? -- bk From bburke at redhat.com Thu May 27 13:16:25 2010 From: bburke at redhat.com (Bill Burke) Date: Thu, 27 May 2010 09:16:25 -0400 Subject: [rest-practices] List subscription? Message-ID: <4BFE70A9.5040705@redhat.com> Somebody post link to subscribe to list? Sergey, in copy, is a new hire that may be interested in some conversations we have here. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From eoghan.glynn at gmail.com Thu May 27 13:19:35 2010 From: eoghan.glynn at gmail.com (Eoghan Glynn) Date: Thu, 27 May 2010 14:19:35 +0100 Subject: [rest-practices] List subscription? In-Reply-To: <4BFE70A9.5040705@redhat.com> References: <4BFE70A9.5040705@redhat.com> Message-ID: Here it is: http://www.redhat.com/mailman/listinfo/rest-practices On 27 May 2010 14:16, Bill Burke wrote: > Somebody post link to subscribe to list? Sergey, in copy, is a new hire > that may be interested in some conversations we have here. > > -- > Bill Burke > JBoss, a division of Red Hat > http://bill.burkecentral.com > > _______________________________________________ > rest-practices mailing list > rest-practices at redhat.com > https://www.redhat.com/mailman/listinfo/rest-practices > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markmc at redhat.com Thu May 27 13:21:30 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 14:21:30 +0100 Subject: [rest-practices] Relative uris In-Reply-To: <4BFE5BD7.7020201@redhat.com> References: <1274957351.29945.111.camel@blaa> <4BFE5BD7.7020201@redhat.com> Message-ID: <1274966490.3057.10.camel@blaa> On Thu, 2010-05-27 at 07:47 -0400, Bryan Kearney wrote: > What happens if you do JSON over a message bus? Are you using the same > serialization? That may be an edge case, but I would not want ot have to > manage multiple serialiing layers per transprot. You're marshalling objects over QMF which contain URIs into the REST API? You'd probably be better with absolute URIs for that, but you could include the entry point URI somehow so that relative URIs could be resolved. Cheers, Mark. From markmc at redhat.com Thu May 27 13:24:43 2010 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 27 May 2010 14:24:43 +0100 Subject: [rest-practices] ID as an attribute [was Re: [deltacloud-devel] Towards declaring the Deltacloud API stable] In-Reply-To: <4BFE5DD1.3070308@redhat.com> References: <1273105038.7253.53.camel@avon.watzmann.net> <1273178876.27025.25.camel@blaa> <1273657470.3062.58.camel@blaa> <4BEA9FD6.3020208@redhat.com> <1274956156.29945.97.camel@blaa> <4BFE5DD1.3070308@redhat.com> Message-ID: <1274966683.3057.12.camel@blaa> On Thu, 2010-05-27 at 07:56 -0400, Bryan Kearney wrote: > > It'd be nice if we avoid letting JSON mapping issues dictate the XML > > design (and vice versa) ... > > > > Is there any way to easily customize the JSON mapping so that these > > attributes can be serialized like other properties? > We just moved to jettiso, so I will take a look. I thought James said you're moving to Jackson? > I know on our side.. > JSON is the preferred transport.. XML is the subordinate. I assume for > you guys it is the other way around? Right, we're defining everything in terms of the XML schema. The JSON and YAML formats will be defined as generic mappings from the XML. Cheers, Mark. From sberyozk at redhat.com Thu May 27 13:25:42 2010 From: sberyozk at redhat.com (Sergey Beryozkin) Date: Thu, 27 May 2010 09:25:42 -0400 (EDT) Subject: [rest-practices] List subscription? In-Reply-To: Message-ID: <913771228.463301274966742497.JavaMail.root@zmail01.collab.prod.int.phx2.redhat.com> Hi I'm already subscribed - thanks Sergey ----- Original Message ----- From: "Eoghan Glynn" To: "Bill Burke" Cc: rest-practices at redhat.com Sent: Thursday, May 27, 2010 2:19:35 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [rest-practices] List subscription? Here it is: http://www.redhat.com/mailman/listinfo/rest-practices On 27 May 2010 14:16, Bill Burke < bburke at redhat.com > wrote: Somebody post link to subscribe to list? Sergey, in copy, is a new hire that may be interested in some conversations we have here. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com _______________________________________________ rest-practices mailing list rest-practices at redhat.com https://www.redhat.com/mailman/listinfo/rest-practices _______________________________________________ rest-practices mailing list rest-practices at redhat.com https://www.redhat.com/mailman/listinfo/rest-practices From bburke at redhat.com Thu May 27 13:25:45 2010 From: bburke at redhat.com (Bill Burke) Date: Thu, 27 May 2010 09:25:45 -0400 Subject: [rest-practices] Securing intermediaries/generic services Message-ID: <4BFE72D9.20102@redhat.com> On JBoss side, I'm defining/creating some RESTful interfaces for our Transaction Manager and HornetQ Messaging. One thing that needs to be defined for this is the authentication mechanism for these intermediary/generic services. For example, the Messaging Service will want to push messages over HTTP to a registered receiver. The TM would want to coordinate registered TX participants. Basically, these intermediaries do work, on behalf of another client. How do you think Enterprise IT would want to authenticate this? Have a centralized identity management server that all servers share and just register usernames to run as with the generic intermediaries? A lease-based protocol where temporary tokens are generated and exchanged? We looked a little bit at OAuth, but it looks more and more like it wasn't made for this kind of interaction. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From bkearney at redhat.com Thu May 27 13:57:31 2010 From: bkearney at redhat.com (Bryan Kearney) Date: Thu, 27 May 2010 09:57:31 -0400 Subject: [rest-practices] Relative uris In-Reply-To: <1274966490.3057.10.camel@blaa> References: <1274957351.29945.111.camel@blaa> <4BFE5BD7.7020201@redhat.com> <1274966490.3057.10.camel@blaa> Message-ID: <4BFE7A4B.6050705@redhat.com> On 05/27/2010 09:21 AM, Mark McLoughlin wrote: > On Thu, 2010-05-27 at 07:47 -0400, Bryan Kearney wrote: >> What happens if you do JSON over a message bus? Are you using the same >> serialization? That may be an edge case, but I would not want ot have to >> manage multiple serialiing layers per transprot. > > You're marshalling objects over QMF which contain URIs into the REST > API? We just added HornetQ for capturing events, so we will get AMQP "soon". We are serializing some objects into JSON there... I am loathe to write specialized serializers. Well.. loathe to write any to be honest. -- bk From sberyozk at redhat.com Thu May 27 13:54:05 2010 From: sberyozk at redhat.com (Sergey Beryozkin) Date: Thu, 27 May 2010 09:54:05 -0400 (EDT) Subject: [rest-practices] Relative uris In-Reply-To: <749168180.466921274967833532.JavaMail.root@zmail01.collab.prod.int.phx2.redhat.com> Message-ID: <268306499.468861274968445051.JavaMail.root@zmail01.collab.prod.int.phx2.redhat.com> Hi you might want adding xml:base attributes then. For example if it were an atom feed containing the relative link URIs then a generic reader may be able to recognize xml:base and properly construct an absolute URI. Json data may indeed contain a base attribute (as opposed to element) too.... You might also want to check how relative URIs with/without xml:base are supported by browsers, ex, in cases where an HTML-based representation is returned. I think I might've heard people saying it depends on the concrete browser implementations. cheers, Sergey ----- Original Message ----- From: "Mark McLoughlin" To: "Bryan Kearney" Cc: "rest-practices" Sent: Thursday, May 27, 2010 2:21:30 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: [rest-practices] Relative uris On Thu, 2010-05-27 at 07:47 -0400, Bryan Kearney wrote: > What happens if you do JSON over a message bus? Are you using the same > serialization? That may be an edge case, but I would not want ot have to > manage multiple serialiing layers per transprot. You're marshalling objects over QMF which contain URIs into the REST API? You'd probably be better with absolute URIs for that, but you could include the entry point URI somehow so that relative URIs could be resolved. Cheers, Mark. _______________________________________________ rest-practices mailing list rest-practices at redhat.com https://www.redhat.com/mailman/listinfo/rest-practices From bburke at redhat.com Thu May 27 14:02:04 2010 From: bburke at redhat.com (Bill Burke) Date: Thu, 27 May 2010 10:02:04 -0400 Subject: [rest-practices] Relative uris In-Reply-To: <4BFE7A4B.6050705@redhat.com> References: <1274957351.29945.111.camel@blaa> <4BFE5BD7.7020201@redhat.com> <1274966490.3057.10.camel@blaa> <4BFE7A4B.6050705@redhat.com> Message-ID: <4BFE7B5C.5020406@redhat.com> Bryan Kearney wrote: > On 05/27/2010 09:21 AM, Mark McLoughlin wrote: >> On Thu, 2010-05-27 at 07:47 -0400, Bryan Kearney wrote: >>> What happens if you do JSON over a message bus? Are you using the same >>> serialization? That may be an edge case, but I would not want ot have to >>> manage multiple serialiing layers per transprot. >> >> You're marshalling objects over QMF which contain URIs into the REST >> API? > > We just added HornetQ for capturing events, so we will get AMQP "soon". > We are serializing some objects into JSON there... I am loathe to write > specialized serializers. Well.. loathe to write any to be honest. > FYI, I'll have a restful interface for hornetq soon if you're interested. You'll be able to mix and match from JMS/HornetQ java apis and have automatic serialization/deserialization so long as a content-type and JAX-RS provider can be figured out. I'm still a few weeks away though... -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com From sberyozk at redhat.com Fri May 28 11:19:53 2010 From: sberyozk at redhat.com (Sergey Beryozkin) Date: Fri, 28 May 2010 07:19:53 -0400 (EDT) Subject: [rest-practices] Securing intermediaries/generic services In-Reply-To: <1532369174.583211275044696849.JavaMail.root@zmail01.collab.prod.int.phx2.redhat.com> Message-ID: <1022005372.583691275045593362.JavaMail.root@zmail01.collab.prod.int.phx2.redhat.com> Hi I've posted a response to resteasy-dev[1] to facilitate some feedback there from Stef/others, here's a summary. I believe there are two main cases. One is when message subscribers/transaction participants 'own' their own servers. This case is the simplest one and I feel is quite likely to occur in practice. In this case an intermediary such as Messaging Server (MS) does authenticate itself with the server indirectly, by creating a signature and thus showing the knowledge of the shared secret generated during the subscriber's secure registration process. MS might also include some sort of enterprise-wide ID so that the server can properly authenticate it but I reckon this would a redundant piece of information given that the subscriber has a trust in MS, by using HTTPS and by owning the server. Another case is where entities doing the subscriptions are actually not owning the servers where MS will push messages too. This is a more complicated, kind of public internet (or very large enterprise) specific case which fits better into the OAuth picture. Identity Management systems can definitely help here so that subscribers can avoid providing name/password to MS and the server and with MS having to know how to authenticate with individual servers. Some opaque IDS such as OpenId/etc can be used to get all involved authenticated and for MS to get OAuth tokens from the servers. Alernatively the subscribers can upload the public keys of the servers to MS and MS will just use HTTPS. Both cases are doable but I'd really like to focus initially on the case 1 during the next weeks Comments are welcome Sergey [1] http://sourceforge.net/mailarchive/message.php?msg_name=AANLkTimhMA9-TDAumm84qLk9yljil6UpymPPkxcHnUxs%40mail.gmail.com ----- Original Message ----- From: "Bill Burke" To: rest-practices at redhat.com Sent: Thursday, May 27, 2010 2:25:45 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: [rest-practices] Securing intermediaries/generic services On JBoss side, I'm defining/creating some RESTful interfaces for our Transaction Manager and HornetQ Messaging. One thing that needs to be defined for this is the authentication mechanism for these intermediary/generic services. For example, the Messaging Service will want to push messages over HTTP to a registered receiver. The TM would want to coordinate registered TX participants. Basically, these intermediaries do work, on behalf of another client. How do you think Enterprise IT would want to authenticate this? Have a centralized identity management server that all servers share and just register usernames to run as with the generic intermediaries? A lease-based protocol where temporary tokens are generated and exchanged? We looked a little bit at OAuth, but it looks more and more like it wasn't made for this kind of interaction. -- Bill Burke JBoss, a division of Red Hat http://bill.burkecentral.com _______________________________________________ rest-practices mailing list rest-practices at redhat.com https://www.redhat.com/mailman/listinfo/rest-practices