The products that excite us today are simple.

Take Apple’s sleek product designs, the negative space of the Google search page, or the single function apps we use on a daily basis.

The idea of erasing clutter permeates into the discussion surrounding APIs or application programming interfaces as well. For example, SOAP has time and time again been labeled as complex, motivating developers to turn to RESTful designs. In the API space, functionality, architecture, and more may be constrained in the wake of simple design ideals.

Though a trend toward simplicity has encapsulated the tech product and web design community, we must not let it have a bewitching effect. For many APIs, complexity is an inherent ingredient critical to function and survive in a competitive market. To find a balance then, when designing APIs for a developer consumer we must constantly remember that complexity is the price we pay for utility.

As design and developer experience are critical components to a successful API, we must take this a step further and ask ourselves who is paying the price for complexity in our APIs? Complexity is inherent to software — Tesler’s law of conservation of complexity states that every application has an inherent amount of irreducible complexity. So the only question then is: who will have to deal with it?

Complexity Within the API Domain

In the discussion surrounding APIs, the idea of complexity resurfaces quite often. According to Ronnie Mitra of API Academy, complexity possesses the following attributes:

• Composed of many parts
• Intricate, elaborate, interconnected
• Difficult to process, resource intensive

There are also differing categories of complexity. Interface Complexity and System Complexity, for example, offer varying perspectives on how to approach API design — from a user experience perspective and from an architecture perspective.

Interface Complexity

Great minds have explored design principles to describe our mentality towards cognitive complexity in product design. In Living With Complexity, Donald A. Norman posits that complexity is a natural element of nature, and should not itself be extinguished:

“Complexity by itself is neither good nor bad: it is confusion that is bad.”

Within The Laws of Simplicity, John Maeda reaches a similar point, saying:

“Simplicity is about subtracting the obvious and adding the meaningful.”

Interface complexity is a direct example of how simplicity and complexity butt heads. Compare two steering wheels — one designed for a Mercedes Benz, and another for a Formula 1 race car. The standard Mercedes Benz steering wheel is less complex — manufacturers privilege usability and aesthetics to cater to the needs of an average driver. Within a Formula 1 race car, on the other hand, many buttons compete for space, triggers and the wheel shape are optimized for intricate functionality — an intentionally complex design epitomizing cognitive complexity.

Complexity thus becomes more about perception. A problem only arises when a complex item is, in it’s unique perception and use case, still conceived as confusing.

Managing Complexity Within API Design

So what do these design principles mean for APIs? Mitra contends that it’s the API designer’s job to manage complexity by improving learnability and usability, and by reducing confusion. This doesn’t necessarily mean that complexity itself needs to be sacrificed.

“Design the interface, but embrace the complexity inherent in your business domain”

Paying The Price for Utility

Complexity is a price you pay for utility. In yet another example Mitra pits the single click Apple mouse against the ultra action gamer mouse — which will win? Well, having a single click is simple — yes, but it is limited in it’s utility, as added complexity allows additional functionality.

In his paper entitled “No Silver Bullet,” Fred Brooks notes that complexity within software should be thought of as “an essential property not an accidental one. Hence descriptions of a software entity that abstract always its complexity often abstract away its essence.”

So, with the goal to create more elegant, user-friendly APIs, what should we be looking for? The answer is to search for accidental complexity — superfluous complexity that can be removed and may be completely unnecessary.

Designing API As a Product: Imitation & Pagination

Design involves deciding whether you will pay the price as a provider, or leave it for the client app developer. This involves picturing the API product lifecycle as a whole to track how complexity arises and is dealt with throughout the main life stages.

An API’s analysis stage begins with identifying objectives and sketching out a plan for success. Conway’s law states that the business systems of communication often mimic the structure of the organization. To match this level of complexity, Mitra recommends that during preparation we consider the second abstraction — that interfaces (APIs) often mimic these systems.

Developers deal with application complexity of the environment they work in. An example of complexity in design is found in API pagination. In a API designed with pagination, when you send a request to the API, the responses spits back in manageable chunks. In a strategy without pagination, however, returning loads of unstructured data means that the client application developer pays the price for complexity.

Considering approaches to pagination, if we take an offset and count methodology that uses input which the client provides, the developer still needs to maintain a good amount of complexity. If we use a fixed page size pagination style, where by default a paged chunk of data with links and hypermedia is returned from a request, less stress is placed on the client. Moving further, smart pagination could involve the right amount of data returned for your specific needs based on your app, the requests you’ve made in the past, and other variables. Mitra stresses to have empathy for the developer and consider the best pagination design that delivers the least amount of unwarranted complexity.

System Complexity

Consider a flock of birds. Can you predict how the entire flock flies when it’s together? In this situation complexity is nonlinear and difficult to predict. Mitra notes that there are both Complex Physical Systems, such as the complex weather systems, and Complex Adaptive Systems, made up of not elements, but agents who change their behavior over time. An example of a complex adaptive system is the stock market. From the viewpoint of complexity, humans can make irrational decisions, and their behavior is changed in the market. The system becomes more difficult to predict, and an equilibrium becomes harder to reach.

Agent Structure

John Holland, an American professor and pioneer in genetic algorithms, describes a theory on agent structure with varying levels of coherence. Agents first determine their performance based on a certain decision — they will buy stock. Second, they review their behavior and assess it, known as Credit Assignment, which happens over time and is scored. This leads to Rule Discovery, wherein agents are led to come up with new rules based on what they’ve learned after experimentation.

Emergent and Adaptive Behavior

Within a system like the stock market, we begin to see specialization and diversity as agents specialize in performing certain tasks. We notice that behavior becomes decentralized — the system as a whole has it’s own behavior, and can adapt to change — the system as a whole demonstrates emergent behavior.

Defining emergence:

“The action of the whole is more than the sum of the actions of the parts”

With this in mind, Mitra suggests developers create systems that consider the adaptive behavior of consumers, and also to extend what we consider a boundary. Instead of designing systems with the notion that the client app is a completely separate entity, Mitra foresees continued decomposition of application boundaries, increasing our ability to change constantly and exhibiting true emergent behavior.

OAuth 2.0 & Complexity

Is OAuth 2.0 complex? Many would say it certainly is. Within the 2.0 version, much utility has been added. The framework contains four flows, a vast amount of moving parts — it is inherently complex. Yet, OAuth is a de facto part of the API security stack. So why is complexity OK in this case?

One way to understand this is to imagine if you could measure the distribution of complexity of OAuth 2. What we see is that the person who pays the price for complexity is the Server implementer, not the client. Suddenly, the integration time and cost is justifiable as the provider pays the price. Leaving a security implementation unresolved may seem like a small problem, yet users will pay a detrimental price for the utility. According to Mitra:

“Design and architecture suddenly becomes all about finding what the essential complexity is, and then moving it around.”

Complexity in Microservices

The traditional monolithic structure is being decomposed, separated into smaller microservices. In a monolith, code maintainers are forced to perform sweeping and elaborate updates across an entire system, increasing the price they pay and the complexity involved. It can be argued that the role for system maintainers, on the other hand, is decreased, as holistically, the system has less pieces to deal with.

Within microservices architecture the inverse occurs and system maintainers now pay the price. Learnability improves, code changes are in smaller batches and thus more fluid, yet the system maintainer how has additional moving parts to deal with.

With containerization, deployment, and discovery now prominent in microservices architecture, a large transition occurs and trade off is made. A service will become less confusing, but holistic system complexity increases. We’re seeing that this complexity is worth paying the cost for.

So, All Good in Microservices?

Mitra contends that a caveat of the microservices architecture is found during implementation. Perhaps this approach doesn’t think enough about the client application. There may be varying data models between microservices, increasing the type and the number of calls being made. This complexity comes at a cost to client app developers — not what you want if you’re developing APIs within a competitive market.

To avoid this, providers are shifting complexity to the SDK. But is this an end all result? Mitra believes that the process often doesn’t erase system complexity but expands the system boundary, creating additional dependencies and system complexity for the user.

Conclusion: Adaptive Complexity as a Feature Set

API consumption is a unique process involving the interaction between the human — exhibiting non-runtime activities and adaptive behaviors — and the machine — with runtime activities and a deterministic behavior. With so many variables at play, Mitra foresees scalable complexity as a future destination for API designers. To sum up our points on complexity remember the following lessons:

• Complexity is natural part of life and is especially inherent to software processes: embrace it!
• Avoid confusion on the client side by improving learnability and usability.
• Use smart pagination to expose data in a way that will make lives easier.
• Hide complexity on the server side whenever possible.
• Have an open outlook to the future of decomposition, and consider how to incorporate system complexity theories within API design.