This is the third article on a series that aims to put the focus of Web APIs back on the Web, on its underlying architecture and on what it means to build evolvable APIs for it. In this first we introduced the architecture of the Web along with its first pillar — Identification. The second and previous article focused on the second pillar — Interaction and the HTTP protocol. The present articles ends this series with an analysis of the third – Formats.

The first, and perhaps most prevalent, format used on the Web is HTML (HyperText Markup Language). However, that doesn’t mean that the Web and its interaction protocols are bound to one specific format. On the contrary, one of the distinctive features of HTTP is the ability to use multiple formats for the transferred representations. This is in sharp contrast with the SOAP’s “one size fits all” characteristic of using XML for everything. This design choice has two important advantages.

Format Flexibility

First, it allows the Web participants to choose the best formats to represent different kinds of information. For instance, using XML to represent a compressed image is awkward. On the other hand, using the JPEG format to represent structured data would also be very far from ideal.

Second, the fact that the Web does not depend on a closed set of formats enables its evolution via the creation and adoption of new formats. A remarkable example is the creation and wide adoption of the JSON format to represent structured data, replacing XML for many purposes.

However, this format independence does not mean that formats are a second class citizen of the Web or of a Web API design process. In fact, it is quite the opposite. The HTTP protocol defines specific metadata headers to convey both the format used (the Content-Type entity header) and the format preferences (Accept request header).

These headers use media type identifiers, such as application/json, composed by a top-level type (e.g.application) and a subtype (e.g.json). These identifiers are managed by IANA, the Internet Assigned Numbers Authority, who runs an extensible registry with all the formats usable on the Web, as well as their identifiers, syntax, and semantics. A quick analysis of the media types present in this registry shows the richness of formats currently available.

Format Flexibility in the Realm of APIs

This format flexibility also applies to Web APIs, which don’t need to be restricted to using JSON-based representations. For instance, if the main client-processing is to produce a printed version of the information, it might be perfectly adequate for an Web API to use a PDF based representation.

Even when using JSON-based representations, Web APIs should avoid the creation and proliferation of ad-hoc formats and instead take advantage of more purpose-specific formats. A great example is the Problem Details for HTTP APIs draft specification, which defines a machine-readable way to represent error details in a HTTP response. The following example, taken from that draft specification, illustrates the use of this new format to convey information about a request with invalid information.

HTTP/1.1 400 Bad Request
Content-Type: application/problem+json
Content-Language: en

{
    "type": "https://example.net/validation-error",
    "title": "Your request parameters didn't validate.",
    "invalid-params": [ 
        {
            "name": "age",
            "reason": "must be a positive integer"
        },
        {
            "name": "color",
            "reason": "must be 'green', 'red' or 'blue'"
        }
    ]
}

Taking advantage of these emerging formats presents multiple benefits for the Web API designer and developer. First, they provide reasonable well architected designs for common problems, based on real world implementations and expert contributions. Most probably, an ad hoc solution will not be as robust and evolvable as these designs. Secondly, by using common formats, developers take advantage of existing code libraries to handle them.

Hypermedia and Links

The Web is not just a set of disconnected resources. It’s the connections between the resources — the links — that make the Web deserve the name it has.

Links are present in representations and provide the client with information on how and when to interact with related resources, identified by their URIs. For instance, an HTML representation of a resource typically contains:

• Links to external resources that should be embedded in the HTML document, such as Javascript excerpts and CSS stylesheets, which are followed before the page is rendered.
• Links to related resources (e.g. a HTML elements), followed when the anchor elements are selected.
• Forms that trigger interactions with other resources, based on information provided by the user (e.g. form HTML elements).

More precisely, a link is a typed connection between two resources, and is defined by:

• the context resource, implicitly defined by the representation where the link occurs;
• the target resource, explicitly identified by a URI;
• the relation type, defining the connection type;
• other target attributes, such as the HTTP interaction method that should be used when activating the link.

RFC 5988 (Web Linking) defines a set of common relation types, such as first, next, self, up and edit. A more complete set of link relations is maintained by IANA at their link relations registry. RFC 5988 also defines a way to define links in the representation metadata, and not on the representation itself, via the Link HTTP entity header. An example of this feature is the pagination information returned by the GitHub API.

Link: <https://api.github.com/user/repos?page=3&per_page=100>; rel="next",
  <https://api.github.com/user/repos?page=50&per_page=100>; rel="last"

Notice the usage of the next and last link relations defined by RFC 5988. The Link header is particularly useful when using links with formats that don’t support them natively and aren’t easily extensible, such as image formats.

Hypermedia Types for Web APIs

On the other hand, the formats, such as HTML, that contain native support for representing links, are called hypermedia types. In the realm of Web APIs, unfortunately the prevalent JSON format is not a hypermedia type because it does not natively provide ways for representing links.

However, JSON does provide enough extensibility to add those links on top of its model. For instance, the GitHub API does exactly that, by using *_url properties to represent links (see https://developer.github.com/v3/#hypermedia)

Fortunately, recently there have been more established proposals to create JSON-based hypermedia formats. In the following sections, we’ll examine three of the top emerging hypermedia types: HAL, Siren, and Collection+json.

HAL Hypermedia Type

One of the first hypermedia types was the Hypermedia Application Language (HAL), which has been adopted by several APIs including the Amazon API Gateway REST API. The following excerpt illustrates the use of HAL for representing issues and comments on a fictional issue tracking system:

{
    "_links": {
        "self": { "href": "/issues/123"},
        "author" : {"href”: “/users/pmhsfelix"}
    },
    "title": “NordicApi slides",
    "description": "Create slides for the Nordic APIs session",
    "_embedded": {
        "issues:comments": [
            {
                "_links": { 
                    "self": {"href" : "/api/issues/123/comment/1"}
                },
                "text": "don’t forget it’s only 25 minutes"            
            },
            {
                "_links": { 
                    "self": {"href" : "/api/issues/123/comment/2"}
                },
                "text": "finally done"   
            }
        ]                    
}}

Each representation is defined as a top level JSON object with the reserved _links property containing a set of links. The value for this property is an object, whose property names are link relations (e.g. self and author) and the property names are link objects or arrays of link objects.

A link object represents a link via a mandatory href property, with the link target identifier, and other complementary link properties such as title and type. The resource state is represented by a set of custom properties in the root object, such as title and description in the previous example.

The HAL format also has the concept of embedded representations — partial representations of resources related to the root. These representations are contained in the _embedded property, whose value is an object. Each property of this object represents a link relation, and its value is the partial representation for the resource(s) that is the target of the link.

Before a client uses a link to perform a request, there a number of questions that must be answered, namely:

• What should be the request method?
• Should the request have a body and if so how should it be built?

When using HAL, the client only has the link relation to help answer these questions since no other relevant information is provided. This means that there must be a hardcoded contract stating how each link relation maps to a request.

Siren Hypermedia Type

Siren is another emerging hypermedia type which overcomes these limitations by introducing the concept of unsafe links, called actions, as well as actions fields, similar to HTML’s form fields.
The following JSON excerpt, taken from https://github.com/kevinswiber/siren, illustrates how an action is represented:

{
  "name": "add-item",
  "title": "Add Item",
  "method": "POST",
  "href": "https://api.x.io/orders/42/items",
  "type": "application/x-www-form-urlencoded",
  "fields": [
    { "name": "orderNumber", "type": "hidden", "value": "42" },
    { "name": "productCode", "type": "text" },
    { "name": "quantity", "type": "number" }
  ]
}

Notice how the action defines the HTTP method, the request body format, as well as its constituent fields (orderNumber, productCodeand quantity). By making this information explicit in the link control and not hardcoded to the link relation, the Siren media type facilitates API evolvability. An example would be the addition of a new field, which can be dynamically expressed on the link information itself.

Collection+json Hypermedia Type

Finally, collection+json is yet another JSON based hypermedia type. However, instead of being general-purpose, it was designed specifically to represent collections of items, as well as defining interactions with these collections. Namely, it defines support for the following:

• query templates: links controls that specify how to search collections.
• write templates: links controls defining how to add items to collections.

Conclusion: Evolvable Web APIs Series Wrap-up

Defining the formats used in a Web API is one of the fundamental tasks faced by the API designer. Just taking the internal objects and using a JSON serializer to expose them via HTTP is almost never a good decision, especially since more robust formats exist. Take advantage of the format richness that is emerging around Web APIs, namely purpose-specific formats such as problem+json and collection+json, as well as general purpose formats such as HAL and Siren.

The Web is a global information space with unprecedented success, scale and evolvability. Much of these characteristics derive from its core architectural principles. Throughout the Designing Evolvable APIs for the Web series, we’ve highlighted these principles — now take this knowledge to create APIs that are first-class citizens of the Web.

Designing Evolvable APIs for the Web Series