Restful Objects criticism
The Restful Objects spec drew some criticism on twitter today. I’m quite happy to get feedback so long as its constructive. Of course, the ‘net being the ‘net, that ain’t always the case. Oh well. [UPDATE: the replies to this post so far *have* been constructive... my thanks]
Since 140 characters isn’t really enough to drill down into issues and concerns, I thought I’d post some of the criticism here and attempt to answer the points. It’d be nice to think I may get some comments against this post which could help develop the spec, rather than summarily dismiss it. Whether that happens, we shall see.
Anyway, on with the criticism…
@serialseb: Custom headers, with an X- profile, confusing operations and resources, generic media types, use of more custom headers, ignoring Link…
Taking each in turn:
a) The custom request headers defined by the RO spec are:
- X-Page, X-Page-Size
- X-Sort-By
- X-Follow-Links
- X-Validate-Only
- X-Domain-Model
I can imagine that there might be a standard way to specify paging and sorting; if there is I’d be interested in being pointed to the relevant prior art. On the REST project I’ve been working this last year, we deal with pagination using regular query args. The issue I have with that is that it doesn’t distinguish between arguments that could be consumed by the infrastructure (for paging etc) vs arguments that are specific to business logic.
An alternative approach might have been to define standard query args, and require any implementations of the spec to ensure that collisions with business logic are managed some how. That didn’t feel particularly pleasant, though.
The X-Follow-Links is used by clients to provide a hint for avoid the N+1 problem that ORMs can suffer from. To me this seems more elegant than having to go to all the bother of defining a specific resource acting like a view-model which is just there to perform a specific aggregation (eg Order plus all OrderItems). Instead, the client can specify a value of ”items” for this header when requesting the Order resource; this hints to the resource for the returned representation to inline representations of all OrderItems also.
The X-Validate-Only is a hint to request that a mutation to the state of the resource be validated but not applied. For example, a client can verify if a set of arguments for an operation are valid prior to actually performing the operation. This could be useful for some client’s that want to provide feedback in their UI.
An alternative approach would have been to define separate subresources that would allow a client to request validation, and earlier versions of the spec did toy around with this. However, taking this approach added a lot of complexity for a requirement that I felt was more easily satisfied through this custom header.
The X-Domain-Model is another hint for those clients that want to obtain a full representation of the domain metamodel. I suspect this would only really be of interest to those writing a general-purpose RESTful client that provides a generic UI for any domain model (ie in the style of naked objects). I suspect that those who don’t get or dig naked objects (yes, there are many, I know) won’t see the point of this header.
b) The custom response headers defined by the spec are:
- X-Representation-Type
This is intended to allow clients to distinguish different representations. This ties in with the criticism made about…
c) generic media types, presumably criticising that the spec says to return “application/json” as the media type for all representations.
Earlier versions of the spec did say to return “application/vnd.xxx+json”, where “xxx” was a unique string representing the type (eg x.Customer or x.PlaceOrderCommand). The issue, though, is that plugins for browsers (eg REST console) don’t recognise the returned representation as JSON. So the X-Representation-Type is a work-around to deal with such pragmatic concerns.
But… if there’s a more standard header that addresses this (or even a plugin for browsers that understands the vnd.xxx syntax), then I’d like to know.
d) I’m not sure what I’ve confused between operations and resources, though I suspect it’s a reaction to the fact that the spec exposes an object’s actions (eg Customer#placeOrder) as two (sub-)resources: one to obtain a description of the action, and one to invoke the action. No apology here. A domain object (especially if that domain object is really a view model) may want to indicate the valid set of values in order to perform some operation. This is what the action description resource provides. The invoke subresource then allows the operation to be performed (GET, PUT or POST depending on idempotency/side-effects etc).
That said, and as I point out in the spec, there is a school of thought that says that only commands should be exposed as RESTful resources… it shouldn’t be Customer#placeOrder, instead it should be PlaceOrderCommand#execute, so that the arguments needed to perform the command are PUT as subresources of the command resouce prior to it being executed. The RO spec does support this as well. But, admittedly, the examples in the spec don’t use this style.
e) “ignoring Link”; the spec does define its own JSON representation for a link between resources. I did search for an existing standard for representing links, but clearly didn’t look hard enough. I’m guessing that the Link spec you mean is
http://tools.ietf.org/html/draft-nottingham-http-link-header-05
? If so, I shall look into it some more.
@serialseb: Use of 412 for things that are not http header preconditions, pagination following none of the original specs…
a) 412. The spec defines to return a 412 if a client attempts to perform an operation which fails server-side validation. For example, if a customer tries to place an order for a negative number of items, that would probably be a validation error for one of the arguments.
Now, according to w3c, 412 should be returned when:
The precondition given in one or more of the request-header fields evaluated to false when it was tested on the server. This response code allows the client to place preconditions on the current resource metainformation (header field data) and thus prevent the requested method from being applied to a resource other than the one intended.
So, yes, there’s probably some justification in this criticism, though the intent is very similar. Of course a simple 400 might suffice, but I’d prefer a more targetted response code. Are there alternative conventions here?
b) pagination… I mentioned above. I’d be happy to adopt a standard if there is one. Again, my searches didn’t seem to show up a clear standard; see for example this thread on stackoverflow. Admittedly that dates from 2009, so perhaps a consensus has since been reached?
@serialseb: Use of Last-Modified but no Etags..
I can see that an ETag would be needed for concurrency control if there is no Last-Modified/If-Modified-Since, but if there are these headers (which is what the spec says) then what benefit does an ETag bring?
@serialseb: Modelling RPC calls as resources (at least that one is discutable slightly)
I’m not sure exactly which RPC calls are being referred to here… it might be a reference to the spec exposing (singleton) domain services/repositories, eg Customers or ProductCatalog, or it might be a reference to the spec exposing domain object actions (discussed above).
@serialseb: And all in all plenty of opaque behavior straight at the http level, disregarding completely defining a correct media type definition.
I’m guessing the opacity is in terms of the custom headers (discussed above); media types vs representation types was also discussed above. To my knowledge there’s nothing else in the spec that might be considered opaque.
Indeed, I should turn that accusation around. The fact that the spec allows any set of domain objects (be they entities, commands, view models or whatever) be exposed in a completely predictable manner, and defines how to expose the metamodel of those domain objects, and as such could be consumed by a completely generic REST client if required, makes me think that the representations and resources defined by the spec are the exact opposite of being opaque.
In contrast, I would describe a REST system as opaque if its resources are designed in an adhoc fashion, are poorly documented (because who wants to do documentation?), that doesn’t fully define how to handle bad inputs, where the provided representations don’t explain fully how to follow the links, where there’s no guarantee that all links needed to fully traverse the resources will even be provided in representations, and where the underlying domain model is at best implicit.
@darrel_miller: “So, irrespective of which view you have of how to do REST” contains two links to articles saying don’t do what he is doing!
The spec doesn’t actually say which style of REST to do: you can expose view models or commands (a la Rickard Oberg) or entities or whatever. If there’s a failure in the spec, I think it is that it only has examples of entity representations.
The more significant of the two articles I quote is Roy Fielding’s 2008 post on REST APIs being hypertext driven. The representations defined by the spec are very much hypertext driven; if they weren’t then one couldn’t write a generic REST client a la naked objects.
Re-reading Fielding’s post, the place where the RO spec comes closest to falling foul is where he says:
A REST API should never have “typed” resources that are significant to the client…. The only types that are significant to a client are the current representation’s media type and standardized relation names.
One could argue that exposing entities without projection means that those entity types are significant to the client. However, there is a degree of abstraction even here in that the RO spec defines a media type along with the X-Representation-Type header workaround (discussed above). Any, in any case, there’s nothing in the RO spec that says you can’t wrap your entities in view models or commands if you so wish.
@colin_jack: You need to project from any non-trivial domain model if you want any sort of API surely?
@darrel_miller: Exactly. Without that projection you are just doing remote objects again. We tried that route, for 10 years.
I’m guessing that “projection” means a view model or command layer in front of entities; in which case, as discussed, you can do this if you want.
I suppose I should point out though the counter-example (for what it’s worth) that you can build non-trivial domain models without doing this projection; ie the naked objects government system in Ireland. Yes, I know it’s the one we always quote, but it’s still being developed on and it’s still churning out a new release every month (first release was 6 years ago), and is still being used by >1000 governmental employees. So I have an existence proof that it isn’t always necessary to project from a non-trivial domain model. And given that’s the case, let me ask the opposite question: how do you know you are adding projections unnecessarily in your systems?
@algermissen: Why the hell do I need a REST interf. to a dom.obj.model? Any clue what prob this solves?
Because, honestly, we shouldn’t be being paid for building REST interfaces. We should, instead, be applying our brainpower to the business domain. The spec aims to define a standard binding for domain objects/services in terms of REST in order that we can do just that. Yes, there are areas where the spec can be improved; indeed, I’d be amazed if that weren’t the case. But I haven’t yet read any criticism that invalidates this basic goal.
OK; that was all the tweets I could find. But there’ll be more flames tomorrow, no doubt.
Posted on September 22, 2011, in Uncategorized and tagged restful objects. Bookmark the permalink. 8 Comments.
a) Using X- for headers is not a good idea if you ever expect anyone to use those headers in a real application. There does exist a registry for application specific headers http://www.iana.org/assignments/message-headers/prov-headers.html
Sebastien covers the issue here http://codebetter.com/sebastienlambla/2011/02/01/minting-new-internet-media-type-identifiers/
The standard way to do pagination in RESTful application is to use the link relations first, last, next and previous as defined here http://www.iana.org/assignments/link-relations/link-relations.xml
I don’t care what your URL looks like.
X-Follow-Links already exists a Hypermedia-Factor called embedded links (http://amundsen.com/hypermedia/hfactor/) and media types like HTML have stuff link rel=”stylesheet” and Img tags that convey this behaviour. If you really want to put it in a header then use a link header.
X-Validate-Links is another item that can be handled by using a link relation to distinguish between your “commit” link and your “validate only” link. Using a different query param value on the links may be sufficient to distinguish between the two.
I’m not sure I understand what X-Domain-Model is for, but usually metadata is better modelled as a distinct resource because it usually has a completely different lifespan and can be easily cached.
b & c) Sure a browser will not understand your custom media type, that’s why the accept header exists, so that your server is aware of what the client understands. If you get a request from a web browser, then return something it understands. If your client can process your custom media type, then make sure the client tells the server that, in the accept header, so it will return it.
d) I’m not really sure what you are trying to say, but I do know that hypermedia uses links and link relations to allow representations to convey the available “actions”. What the URL looks like is kinda moot.
412
a) 400 is intended to be used for validation type scenarios. It is intentionally vague, but that’s why the spec says that you should also include a body with more details. 412 is specifically intended for dealing with update conflict issues.
Etags avoid the problem with a lack of precision in the last modified since date time.
Sharing private types across the wire creates coupling between client and server. The primary goal of REST is to allow independent evolution of client and server over the long term. Therefore by violating this constraint, you are defeating the purpose of REST. You are possibly creating a completely viable alternative distributed system architecture, but you are not going to induce the characteristics of a RESTful system.
I agree that we should not be wasting our customer’s money by creating unnecessary interfaces. Which is why I use RESTful interfaces to expose UI content and application workflow to remote clients. I have no need to expose domain objects over the wire because my clients are completely ignorant of my domain objects, just like my web browser is ignorant of the banking domain and yet still allows me to do banking transactions.
As Rickard Oberg tries to explain, exposing domain objects via a RESTful interface is a contradiction of objectives no matter how easily your infrastructure allows it to be done.
Darrel,
Thanks for taking the time for the detailed and useful response.
a) For pagination, I’ll look into using link relationships as you suggest rather than X- headers. Is there something similar to allow the X-SortedBy to be removed.
With x-follow-links, the page you quote is about different links from one representation to another. That’s fine, but doesn’t relate to what follow-links is trying to achieve: a hint from client to server that the contents of links are followed by the server resource in order to create an aggregate representation (eg Order + all OrderItems). It seems like an obvious thing to want to do (we could made use of it several times on the project I’ve worked on this year).
Per x-domain-model, we only introduced it to allow implementations to evolve from a very simple form of expressing metadata to a more formal style where metadata resources are separate (and cached, as you say). Part E of the spec defines these resources.
b) & c) media types… ok, what you’ve said makes sense, though again I found that plugins such as REST console don’t set the Accept: application/json header and so an absolutely strict handling here would bump up against reality. But, yes, it would make sense to define application/vnd.ro-domainObject+json as a media type instead of plain application/json along with an X-Representation-Type of “domainObject”; ditto for the other representation types (objectProperty, objectCollection, objectAction, list etc), and return this if the Accept header specifies it is understood by the client. Otherwise, default to application/json.
d) you said: “hypermedia uses links and link relations to allow representations to convey available actions”. +1 That’s what the spec aims to do.
412 … ok; I pushed too hard on this one. I shall revert to plain 400.
etags. ok; I was aware of etags but – stupidly – hadn’t realized that last-modified only specified to the nearest second. Can you tell me: does that mean that most systems use only etags for concurrency control, and the last-modified is really only just informational?
The last issue you raise is about sharing of private types and coupling. Jan Algermissen has also replied to this post and picks up mostly on this point, so I’ll reply in more detail there.
Thx again for the feedback, do appreciate it.
Dan
Dan,
building a REST API (which essentially means “defining the proper media types” because the API (HTTP) is already defined) means creating a service layer. The domain model is an implementation detail that REST deliberately aims to *hide*. IMHO, what you are trying to generalize is the *specific* design work that needs to take place anyhow. Generalizing it just means to shove it up another layer instead of solving it.
Jan,
thanks for taking the time to reply. Darrel also replied, and I wanted to address one of the main points he raised along with yours together.
Darrel said: “sharing private types across the wire creates coupling between client and server. The primary goal of REST is to allow independent evolution of client and server over the long term… I use RESTful interfaces to expose UI content and application workflow to remote clients. I have no need to expose domain objects over the wire because my clients are completely ignorant of my domain objects”
So, let me ask… what media types do you use? In his javalobby article, Rickard Oberg defines two use cases: “/account/changepassword” and “/administration/users/<username>/resetpassword”. I’m presuming you use a similar approach, so what would be the media type for the representations returned by these resources?
In the comments to his article Rickard says: “I use queries and commands as the way to build my REST API, but this is not something a client has to actively understand”. He also suggests that a GET on a resource produces a form, while a POST invokes the form. From that, it sounds like he is using reasonably generic media types. However, they don’t sound any more generic than the media type that RO defines, ie: application/vnd.ro-objectAction+json.
Darrel; you also make the interesting point: “my clients are completely ignorant of my domain objects, just like my web browser is ignorant of the banking domain and yet still allows me to do banking transactions.” What capability is it of web browsers that allows us to do this? Well, it’s because web browsers only understand HTML and PNGs, but they don’t care about the content of the HTML or the picture within the PNG. They are a completely generic viewer. The irony is heavy here, because of course what naked objects tries to do (and yes, has never succeeded in doing) is to provide a completely generic viewer for business domain objects. We do intend to write fully generic viewers against the RO spec, and the existence of these viewers would demonstrate that RO is restful. But RO is in part an attempt to allow developers to write their own custom viewers to access domain objects if that’s what they want. In your reply you indicated that such a design might be “possibly [be] completely viable, but [isn't] RESTful”. Maybe that’s the case; it’d be somewhat akin to screenscraping an HTML page. But if that’s what people want to do, then RO doesn’t try to stop them.
Jan; in your reply you make the specific comments that REST implies there must be a service layer with the domain model is an implementation detail. This service layer sounds like the Rickard’s use cases above, perhaps combined with view models? As I discuss in the spec – but it clearly needs beefing up – there is nothing in the spec to prevent one from doing this. As you rightly then point out, generalizing REST does indeed mean shoving the explicit design work for this service up into another layer. So what’s the benefit?
Well, one benefit is that there’s then a very sharp divide between the infrastructure goo that is all about handling HTTP headers etc vs the business logic of exposing use cases. All the use cases are implemented as simple Java (or .NET) objects; it’s down to the infrastructure to locate these use cases and delegate to them. These use case objects in turn interact and delegate to domain object entities; these don’t have to be exposed if you don’t want them too. From a project process standpoint, I can divide responsibilities in my project team between those who need to understand/enjoy that infrastructure stuff vs those who can focus on the core domain logic. (Rickard’s Streamflow project has similar objectives and benefits, I see. However his project only exposes use cases whereas the RO spec attempts to accommodate a broader church).
Another benefit that leads on from this is testing: I can fully test my business domain object as unit tests without having to fire up a Jetty (or equivalent) webserver instance in order to verify the representations. Keeping the TDD red/green/refactor cycle short is a big deal, I think.
Another benefit is that my business logic is fully documented; I can just run javadoc against my domain objects and the REST consumers can infer what representations will be exposed. This might just be a maturity issue… there aren’t any good tools to generate documentation from RESTful interfaces that I’m aware of (at least, the project I’m on doesn’t use them and it’s been a continual gripe throughout).
Another benefit is that, actually, it may not always be appropriate to hide the domain model. Rickard’s justification for doing so in his article has a lot of hand waving in my opinion; the most I got out of it is that the conditional logic for rendering links based on user permissions is difficult to do. Well, that isn’t the case if one has a domain object metamodel to hand… it’s completely automateable. But in any case, it’s clearly possible to build viable systems that operate directly on representations of entities; we’ve built a REST system in this way this year and it’s perfectly fit for purpose.
Thanks again, both, for your comments. I don’t know whether I’ll have changed your views on the spec, but I’d be interested in your comments nonetheless.
Dan
“The spec aims to define a standard binding for domain objects/services in terms of REST in order that we can do just that.”
One other comment I have is that to make a case in this particular situation you’d be far better with working software.
I’d think it’d be a good idea to take a non-trivial problem you’ve solved before (or want to solve) and use your approach to solving. Then you could make the code public but also let people access the resources publically.
Indeed, and that’s what I’m working on in Apache Isis.
As to a non-trivial application demonstrator, I’m hoping to get permission to reimplement the system I’ve been working throughout this year. But if not then I’ll select something of equivalent complexity.
Dan
Dan, I suggest you take your question to a forum ( best: rest-discuss ) where replying is easier than here. It might also help to break them up in digestable chunks – personally, I am sort of overwhelmed with the length of your reply.
rest-discuss will also offer you a much larger audience.
Jan
Hi Jan,
Someone else pointed me to rest-discuss, also, so thx – will do.
Dan