We’ve discussed in length API business, monetization techniques, and the importance of improving the experience for our target developer consumer. But underneath the hood, what does the architecture of an API platform actually look like? Just as developers who use an API rely on it to power their applications, a similar ecosystem may exist fueling the components that make up an API itself. An API may tap into various services to support its own backend, making the architecture stack deceptively complex.

An API’s architecture often mimics how it is maintained and what the service accomplishes. In this piece we introduce distinct API architectures of varying complexity categorized by function and access level. Guided by Johannes Lundberg, Founder and CEO of 46elks, we’ll define API backend structures based on their reliance to external/internal databases and type of functionality offered, outlining fundamental architecture stacks that composes various APIs.

Four Kinds of APIs

Lundberg sees the essence of an API as:

“turning traditional complexity into programmatic simplicity.”

To follow this maxim, it’s the API provider’s job to disguise complexity when possible to favor ease of use. As design intimately affects behavior, Lundberg organizes APIs by four distinct behavioral criteria. The first two categories deal with the service itself — “data access” and “side effects.”

Data access APIs provide access to retrieve or manipulate any kind of data. Examples include open data initiatives, social media APIs like Instagram or Twitter, or public transportation APIs like OneBusAway. These services may hold a complex database or may be as simple as a single CSV file. Their main value is the exchange of data.

Side effect APIs, on the other hand, are more geared toward offering functionality as a service. The purpose of these APIs are to store data in a cloud setting, facilitate payments online, offer Infrastructure-as-a-service, cloud communications, and more.

You can further categorize web APIs by access type — are dependencies internally or externally controlled? Self contained APIs have all that is needed for the provider to service their API in-house; there is no reliance on external components. This is opposed to vendor dependent, where the software strongly relies on functionality or data from a 3rd party company or government body.

1) Data Access: Self Contained API Architecture

Say you own a crowdfunding application and are exposing a user information API that allows developers to access user account information. It works over HTTP and shoots back JSON format. As you have full control, you can replicate this same backend, and make it scalable for other situations as well. As you own the data and control call limiting, making cloud ready APIs is a simple process with a basic backend composition as such:

2) Data Access: Vendor Dependent API Architecture

Say your application aggregates user accounts from many different crowdfunding platforms to provide a single GUI to an end user. In this scenario, the API provider relies on incoming data from third party crowdfunding platforms. For any API that relies on vendor integration, throughput can become an issue. You also tend to deal more with legacy integrations where incompatibilities may cause disturbance. These factors can be mitigated, but with these dependencies you lose control over call limiting and updates. Because of these factors, the architecture is structured differently:

3) Side Effects: Self Contained API Architecture

Functionality-oriented APIs inherently involve more complicated factors than a simple data access API. For example, if you are a print company that offers an API to automate print orders, you will likely involve machinery in your backend. In a self-contained environment, you control both, and thus involve your product intimately in your API’s backend:

4) Side Effects: Vendor Dependent API Architecture

Services like the Printchomp API may specialize on solely the software component, a middleware connecting the end user to various end integrations. More than requesting data, a service-oriented API with dependencies could involve making an actual phone call, sending a text message, making a payment, analyzing a picture, or more. The backend for a service like this is fundamentally organized as such:

In Reality

We’ve outlined very introductory diagrams that outline potential API backends. In reality, however, an API platform could be far more complex, uniting third party partner systems, internal databases, and self-controlled machinery.

Aligning Feature Set with API Architecture

What is your main feature? Architecture is there to facilitate this purpose. If you are designing an API from scratch and unsure about the core offering, start with defining how the API architecture will be designed according to these four types. If you don’t immediately fit into any of these diagrams, Lundberg advises to take a kernel of wisdom from the startup world: prototype first.

As development costs are high, prototype first instead of designing a robust application with a complex architecture. In an analysis stage, it’s best to start lean with initial user testing, and then iterate based off consumer demand. Have your consumers use your API — what’s beneath the hood matters less to them. Lundberg advises any new API provider create a service that first fulfills the core needs of its users, and to then adjust to meet real market conditions following these steps:

1. Product Design
2. Specifications
3. Maintainability
4. Scalability
5. Feature Creep

”The golden rule of API design: When in doubt, leave it out!”

Lundberg reminds us that when a parameter is established, there is no easy return. No consumer wants to change their code implementations, and an API dependency could be running for 10–15 years. Instead of creating a robust service off the bat, increase operations of your support channels and listen in. Your consumers will get back to you.

Next

After business goals and basic functionality is decided, technical questions naturally arise that effect the architecture. Some examples include:

• XML/JSON: Most of the API development community advises to use JSON.
• Servers vs IaaS: Are you using your own servers or are you outsourcing this data?
• PUSH or PULL: Does your API allow retrieval and manipulation?
• OAuth or Basic: Use advanced security mechanisms to protect your API stronghold.
• Consistency vs Reliability: You can always fix data later, but you can’t buy back two hours of downtime.

Eventual Design

Once you develop a prototype and test, you end up with something like the following schematic. There is a failsafe mechanism that allows for a Backup Data Center, and internal flows include your Load Balancer, Web Servers, Message Bus, Workers, Databases, and background processes like Billing. The API platform as a whole may interact with external vendor data/functionality or internal machinery. If you retain a similar outward offering, the API may not have to change much at all throughout dummy stages to full release, as what you’re doing is making the platform more maintainable and scalable.

46elks Case Study: Necessary Hidden Complexity

In this article we’ve simply scratched the surface, as building APIs should be complex. Take the European 46elks Voice SMS/MMS JSON API. From an outside perspective, all consumers notice is an API platform with stable design and easy accessibility. Under the hood, however, is a very busy schematic full of many necessary interrelations, separate data storages, and back and forth module interactions.

“It’s a lot of stuff to work with and keep running and maintain and keep scalable, and in our case when we have lots and lots of carriers we’re working with we have to make redundant couplings for each of these carriers for each of our data centers. But it’s worth it — this is the complexity you’re removing for your consumer.”

API providers should take complexity and make it appear simple — that is the actual value for the developer consumer, as substantiated by Lundberg and many API practitioners. Hone in on improving architecture as soon as possible, as this devotion will lead to future value. Lundberg imparts the following general advice for API providers:

• Perform versioning.
• Use JSON: There’s very little reason to use XML outwards.
• Talk to your API consumers: prototype, add basic features, then add more based on their response.
• Use the 80/20 rule to cater functionality to the bulk of consumers, rather than all use cases.
• Select vendors carefully: Critically evaluate the maintainer — as with most open source software, there will not be a corporate service with a dedicated support team.