Extra Fox – A blog by Christopher Taylor

Opinionated Guidelines for Designing a Truly RESTful Web Application

Posted in Uncategorized by extrafox on January 6, 2014

It seems that every vendor and startup these days has a “REST API” that they are advertising. Yet, when I examine most of these APIs, they are clearly not RESTful. Most of them are really Remote Procedure Call (RPC) interfaces, but since they are using cacheable HTTP GET requests and other HTTP methods like PUT and DELETE, they call themselves RESTful. While there are advantages to designing RPC style interfaces based on RESTful principles, these APIs are missing out on the full power of a truly RESTful web application design.

Of course, not every web application should utilize REST, and that is fine. If you don’t need to build a truly RESTful web application, then I would direct you to Vinay Sahni’s great set of best practices [2] for building RPC interfaces that call themselves RESTful. In fact, I will be recommending many of the same practices where appropriate and will generally follow the same format for this article.

I have organized this document for both linear and random access. If you’re in a hurry, you can just start with the guidelines and follow the links for detailed explanations. For everyone else, I recommend starting at the top and reading to the bottom.

Guidelines

REST is a network-based software architecture based on specific design constraints

The authoritative source for the Representational State Transfer (REST) software architecture is Roy Fielding’s 2000 doctoral dissertation, “Architectural Styles and the Design of Network-based Software Architectures“. In it he defines a number of different network software architectures that are in common use. He then defines REST as a network software architecture created based on a process where [1],

a designer starts with the system needs as a whole, without constraints, and then incrementally identifies and applies constraints to elements of the system in order to differentiate the design space and allow the forces that influence system behavior to flow naturally, in harmony with the system.

The constraints Fielding identifies are,

  • Client-Server – Allows for a separation of concerns. In particular, allows components to evolve separately
  • Stateless – All state is handled on the client and necessary state is passed to the server with each request
  • Cache – Caches, often times transparent to client and/or server, can be used to improve performance
  • Uniform Interface – Simplifies architecture and improves visibility of interactions
  • Layered System – Allows for improved scalability, performance and security policy enforcement
  • Code-On-Demand – Optional constraint that allows for transfer of executable code

Of all of these constraints, Uniform Interface is probably the most misunderstood and the most fundamental to the REST architectural style. Because it is misunderstood, it is often one of the first constraints to get relaxed or ignored during web application design. The key properties of Uniform Interface are,

  • Identification of Resources
  • Manipulation of Resources through Representations
  • Self-Descriptive messages
  • Hypermedia as the engine of application state (HATEOAS)

Very simply, if you aren’t abiding by these constraints then your web application is not RESTful. It may still be useful and you may have good justifications for making the tradeoffs you make, but there is not point in calling it a REST architecture.

Further Reading: Roy T. Fielding, Architectural Styles and the Design of Network-based Software Architectures [1]

Avoid the tendency to build RPC interfaces and call them RESTful

Most of us come from a background of Object Oriented, Functional and/or Relational Database programming so, when we go to start building web applications, we tend to resort to the paradigms that we are familiar with. As a result, most APIs that claim to be RESTful are really RPC interfaces that use only some of the architectural features of REST. From Fielding [1],

What distinguishes HTTP from RPC isn’t the syntax. It isn’t even the different characteristics gained from using a stream as a parameter, though that helps to explain why existing RPC mechanisms were not usable for the Web. What makes HTTP significantly different from RPC is that the requests are directed to resources using a generic interface with standard semantics that can be interpreted by intermediaries almost as well as by the machines that originate services. The result is an application that allows for layers of transformation and indirection that are independent of the information origin, which is very useful for an Internet-scale, multi-organization, anarchically scalable information system. RPC mechanisms, in contrast, are defined in terms of language APIs, not network-based applications.

The constraints that tend to get relaxed the most are the Stateless and Uniform Interface constraints.

The Stateless constraint is often relaxed to pass session data, usually for the purposes of authentication. This isn’t the worst choice, but it does negatively affect scalability as Servers are required to manage the session state across a cluster.

The Uniform Interface state is almost completely ignored in nearly every so-called REST API out there. There are a lot of reasons for this and many of them are valid. To maintain this constraint, it is important to realize that HTTP already fully defines the interface for Server and Client communication. When semantics are encoded in URIs that need to be understood by Servers and Clients, then this constraint has not been met. When Representations are not Self-Descriptive, then this constraint has not been met. When allowable state transitions are not provided by the Server as links, then this constraint has not been met.

Further Reading: Roy Fielding, REST APIs must be hypertext-driven [18]

Resources are the key abstraction of a RESTful web application

I can’t really say it better than Fielding, so let’s start with this [1],

The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service (e.g. “today’s weather in Los Angeles”), a collection of other resources, a non-virtual object (e.g. a person), and so on. In other words, any concept that might be the target of an author’s hypertext reference must fit within the definition of a resource. A resource is a conceptual mapping to a set of entities, not the entity that corresponds to the mapping at any particular point in time.

Most of your design effort should be concentrated on identifying Resources and formalizing the Representations of these Resources. This is especially true in a RESTful web application, because the Uniform Interface constraint ensures that the interface to the Server is straightforward and doesn’t change from one REST application to another.

Keep in mind that Resources are nouns (not verbs) and they are coarse grained (not fine grained).

Maintain a separation of concerns between the Server and the Client

In the REST architecture, the Server and the Client have clearly defined responsibilities. From Fielding [1],

Separation of concerns is the principle behind the client-server constraints. By separating the user interface concerns from the data storage concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components. Perhaps most significant to the Web, however, is that the separation allows the components to evolve independently, thus supporting the Internet-scale requirement of multiple organizational domains.

The unique responsibilities of Servers and Clients is another of the misunderstood principles underlying the REST architecture. I would even go so far as to say that I think most developers have the responsibilities reversed. This isn’t surprising given that in most programming environments, the developer is building applications that utilize APIs provided by libraries. However, in REST, the Server is the application. The Client is providing views for the user to interact with the application.

Representations need to meet the Self-Descriptive and HATEOAS constraints

Representations are how all state is communicated in a RESTful web application. From Fielding [1],

Depending on the message control data, a given representation may indicate the current state of the requested resource, the desired state for the requested resource, or the value of some other resource, such as a representation of the input data within a client’s query form, or a representation of some error condition for a response.

In order to meet the Self-Descriptive constraint, the format of the Representation must support formal definition of semantics. XML and JSON-LD both have provisions for meeting these requirements, however, JSON-LD is much more developer friendly.

The HATEOAS constraint is probably the constraint that is most unfamiliar to application developers. However, it becomes more clear when you understand the separation of concerns that Fielding has defined. Since the Server is responsible for managing the Identifier namespace (URIs), it is the Server that should define what URIs correspond to different state transitions. Clients are responsible for presentation to the user and handling user input and should only concern themselves with manipulating Representations, not Identifiers.

Further Reading: Parastatidis, et al., The Role of Hypermedia in Distributed System Development [16]

Use JSON-LD as the primary Representation format

REST is an architecture that is in desperate need of some standardization around media types. Of course, the W3C has been working on the XML specifications for over a decade to meet the needs of REST. Unfortuantely, XML has proven to be less popular than JSON for most developers. From Mark Lanthaler et al. [13],

JSON-LD is an attempt to create a simple method to not only express Linked Data in JSON but also to add semantics to existing JSON documents. It has been designed to be as simple as possible, very terse, and human readable. Furthermore, it was a goal to require as little effort as possible from developers to transform their plain old JSON to semantically rich JSON-LD.

With JSON-LD, one of the missing pieces falls into place making it much more feasible to create truly RESTful interfaces. JSON-LD provides makes it possible to meet the Self-Descriptive and HATEOAS constraints while also opening up the power of Semantic Web.

Further Reading: Mark Lanthaler and Christian Gütl, On Using JSON-LD to Create Evolvable RESTful Services [13]

Servers are responsible for managing Resource Identifiers

For HTTP based REST web applications, the Resource Identifier will be a URI. From Fielding [1],

REST uses a resource identifier to identify the particular resource involved in an interaction between components. REST connectors provide a generic interface for accessing and manipulating the value set of a resource, regardless of how the membership function is defined or the type of software that is handling the request. The naming authority that assigned the resource identifier, making it possible to reference the resource, is responsible for maintaining the semantic validity of the mapping over time.

The separation of concerns between servers and clients implies that the client should not need to parse or construct URIs. The internal structure of an URI is an implementation detail for the server and should be opaque to the client. This doesn’t mean that URIs can’t have internal structure, only that this structure is not part of the published interface to the application. Instead, published interfaces should focus on constructing/modifying Representations which are passed between client and server.

API version numbers do not belong in the URI

You will find conflicting opinions on the Web about whether version information for your web application should be included in the URI. When building an RPC style interface where the Server and Client are tightly coupled, assuring that both versions are synchronized becomes a primary concern. When building a truly RESTful web application, the need to maintain versions should be mitigated by adherence to the Uniform Interfaces constraint.

The expectation of a REST Resource Identifier is that the semantics of the Resource that is identified is maintained. Since a Resource as a concept is usually unrelated to the version of the application, these should not be so tightly identified with one another in the URI.

You should be choosing REST because you need to build an “Internet-scale, multi-organization, anarchically scalable information system.” This means that your Representations need to be extensible and your application needs to be resilient to having different Client version out in the wild. Of course, the Web is a perfect example of how this works in practice, which doesn’t mean to say that it always works well.

Further Reading: Robbie Clutton, API Versioning [15]

Use HTTP Status Codes correctly

HTTP Status Codes are a principal kind of Control Data that should be set properly and returned with every request. The complete list of codes and full descriptions can be found in the HTTP/1.1 specification [10] and in RFC 6585 [11]. Here are a few that have particular significance for building RESTful web applications.

  • 200 OK – The request has succeeded
  • 201 Created – The request has been fulfilled and resulted in a new Resource being created. Should include a Location header indicating the URI of the created Resource.
  • 202 Accepted – The request has been accepted for processing, but the processing has not been completed. This is the proper response for asynchronous processes and, like 201, should include a Location header.
  • 204 No Content – The server has fulfilled the request but does not need to return an entity body. This is a good response to a successful DELETE call.
  • 400 Bad Request – The request could not be understood by the server due to malformed syntax.
  • 401 Unauthorized – The request requires user authentication.
  • 403 Forbidden – The server understood the request, but is refusing to fulfill it.
  • 404 Not Found – The server has not found anything matching the Request-URI.
  • 429 Too Many Requests – The user has sent too many requests in a given amount of time.

In addition to status codes, error conditions should contain an entity body that provides further details of the error that are useful for debugging and/or displaying to the user.

Further Reading: IETF, RFC 2616, Hypertext Transfer Protocol — HTTP/1.1 [10]

Provide a useful response for errors

Though HTTP Status Codes provide a basic set of codes for signalling errors, you will also want to make sure to provide a useful response from your application. Error responses should be well structured JSON-LD data just like any other Representation that you would communicate to the Client.

Further Reading: Vinay Sahni, Best Practices for Desigining a Pragmatic RESTful API [2]

Clients manipulate Representations and signal actions by following links

Now we’re getting to the meat of what it means for an application to be RESTful. Again, Fielding [18],

A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

Most APIs define semantics in developer documentation and expect developers to build clients that follow the API semantics. In REST, the media type should define the semantics. When a client receives a Representation as a given media type, it should follow the semantics defined by that media type. The interface to the server should be uniform, i.e. no special rules for parsing or accessing URIs.

Further Reading: PaySwarm Web Payments Specification [19]

Servers provide links within Representations to identify allowable transitions

Need I repeat that REST applications are link driven? No, then I’ll let Fielding do it [18],

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly.

Again, semantics are defined by the media type and this includes how to access other Resources as well as the types of actions that are allowed. These Resources and actions should be referenced as links within the Representation.

Clients should not parse or construct URIs

This is really just a corollary to the statement above, “Servers are responsible for managing Resource Identifiers”. If clients are parsing URIs, then there is tight coupling between the client and the server. Tight coupling makes changes to the application (client and server) difficult as you run the risk of breaking systems that are already in use. So, the server is responsible for creating URIs and the client just follows them.

Resources and References

[1] Roy T. Fielding, Architectural Styles and the Design of Network-based Software Architectures, http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

[2] Vinay Sahni, Best Practices for Desigining a Pragmatic RESTful API, http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api

[3] IETF, RFC 3987, Internationalized Resource Identifier, http://www.ietf.org/rfc/rfc3987

[4] Richardson, Leonard; Ruby, Sam (May 2007). RESTful Web Services. O’Reilly Media. ISBN 978-0-596-52926-0

[5] JSON for Linked Data, http://json-ld.org/

[6] REST API Anti-Patterns, http://www.infoq.com/articles/rest-anti-patterns

[7] Model Your Application Domain, Not Your JSON Structures, http://www.markus-lanthaler.com/research/model-your-application-domain-not-your-json-structures.pdf

[8] The RESTful CookBook, http://restcookbook.com/

[9] Manu Sporny, Why should you use json-ld, https://plus.google.com/+ManuSporny/posts/T5WkpieNrjJ

[10] IETF, RFC 2616, Hypertext Transfer Protocol — HTTP/1.1, http://www.ietf.org/rfc/rfc2616

[11] IETF, RFC 6585, Additional HTTP Status Codes, http://tools.ietf.org/search/rfc6585

[12] Manu Sporny, JSON-LD is the Bee’s Knees, http://manu.sporny.org/2013/json-ld-is-the-bees-knees/

[13] Mark Lanthaler and Christian Gütl, On Using JSON-LD to Create Evolvable RESTful Services, http://www.markus-lanthaler.com/research/on-using-json-ld-to-create-evolvable-restful-services.pdf

[14] StackOverflow.com, Best Practices for API Versioning, http://stackoverflow.com/questions/389169/best-practices-for-api-versioning

[15] Robbie Clutton, API Versioning, http://pivotallabs.com/api-versioning/

[16] Parastatidis, Webber, Silveira and Robinson, The Role of Hypermedia in Distributed System Development, http://www.researchgate.net/publication/221271423_The_role_of_hypermedia_in_distributed_system_development/file/9fcfd50f236cf1ef99.pdf

[17] Curtis Schlak, HATEOAS: A Follow-Up Discussion About REST, http://curtis.schlak.com/2012/01/23/hateoas-a-follow-up-to-rest-for-r33lz.html

[18] Roy Fielding, REST APIs must be hypertext-driven, http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

[19] PaySwarm Web Payments Specification, https://web-payments.org/specs/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: