In a previous article we encouraged every company to join the digital revolution and provide APIs. But when faced with an existing Information System (IS), the steps toward exposing a business process with an API become far more intricate. An IS may be bloated with pre-existing organizational processes and data structures, which, if exposed directly, can make for a very poor user experience, and possibly a complete deadlock.

Therefore, designing APIs to work with a pre-existing IS cannot be taken too lightly. In this article, we’ll analyze what usability factors to consider and what complexities to prevent, proving that providing APIs that work with an existing IS is not a pipe dream doomed to fail.

An Existing Information System is NOT a Good API

An information system can help a company by allowing ways to manage data and business processes. Since APIs are another method for handling data, some architects take the stance that providing APIs equates to exposing the information system as is. However, exposing an information system as it is is not a good idea as the IS can be:

1. Complex
2. Unsecure
3. Slow to evolve

Reason #1: Existing Information System is Too Complex

He is a barbarian, and thinks that the customs of his tribe and island are the laws of nature.

George Bernard Shaw, Caesar and Cleopatra

After years of evolution, an IS can excel at fulfilling its company’s needs. But after all these years of evolution, an IS becomes complex — for outsiders and even insiders — unsuitable for direct exposure as an API. An IS is naturally complex because it mirrors a company’s organization, functional details, technology, and history.

With all this baggage, a consumer becomes annoyed when using an API that relies too much on an internal IS. As a case study, we’ll examine a hypothetical API that allows a customer to place an order to purchase a product with an API that mimics the way an IS might behave.

Organizational Complexity

A company’s IS is a reflection of the company’s organization that most end users do not need to know. When placing an order, an API consumer shouldn’t have to deal with internal organizational details such as:

• Addressing the order to a specific branch. If the shipping address is in Europe, the orderer shouldn’t have to specify addressing the German branch instead of US branch.
• Dealing with processing locations. An order’s shipping will be processed by two departments in the US and one in Germany.

Functional Complexity

Outside users can’t possibly be as familiar with an IS as the employees who have worked for years within a company. Often, even these employees do not know all the company’s functional rules. Therefore, when placing an order, API consumers shouldn’t have to deal directly with functional details such as:

• Details that determine shipping costs. When the order ships from Germany, the shipping cost is based on weight and size, but when shipping from US, the shipping costs is based on weight only.
• Availability based on branch location. If an item is not in German stock but is in US stock, an order can finally be placed through the US branch.

Technological Complexity

A company’s IS may be a motley IS composed of heterogenous components proposing heterogeneous interfaces. This may take the form of various protocols and communication methods (synchronous, asynchronous, HTTP, RPC, RMI, Corba, …) , different types of data formats (XML, JSON, binary, text, …), different representations of the company’s data, and heterogenous error handling. When placing an order, API consumers shouldn’t have to worry about technological details such as:

• Unnecessary extra steps. An order is initiated by calling an XML RPC service that then tells which system to call (American or German).
• Incongruencies across data structures. A European order and an American order do not share the same data structures, naming conventions, controls, or error handling.
• Annoying data entry limitations. A European order is transmitted to the German branch by sending a binary message through MQSeries, whose structure is a reflection of a DB2 table which limits field name to a length of 6 and field value to a length of 32.
• Pesky request processes. An American order is transmitted to the US branch by posting CSV data on a URL.

Historical Complexity

A company’s IS is forged after numerous projects, organizational changes, and technological shifts. These evolutions spawn processes that fulfill the company’s needs, but are a drag from an outsider’s perspective. Immersed within a company’s historical background, the logic of an IS is often not understandable from a user’s point of view. Considering these perpetuated historical details, when placing an order API consumers do not need to:

• Know the reasoning behind specific processes. When transmitting an order to the German branch the number of items is indicated in the productDesignation field because it was faster to use this unused field instead of adding a new one.
• Perform repetitive actions to fit historical request structure. When transmitting an order to the US branch, and due to delivery service splitting in two sub-services, the CSV data has to be posted on two different URLs.

Reason #2: Existing Information System is Unsecure

A ship is safe in harbor, but that’s not what ships are for.

William G.T. Shedd

Sheltered for years in a closed environment can also lead to an IS that is not secure enough for outside exposition. The following can lead to huge security concerns:

• Lack of IAM (Identity and Access Management)
• Risks of inconsistency
• Weak components

Lack of IAM

When exposing an IS to the outside, a company must ensure that it handles identity and access management to authorize and authenticate users as well as their permissions across all interactions with the API. It wouldn’t be safe to give full access to the company orders list to any API consumer, allowing any consumer to access or modify any order. Rather, an API should expose only what needs to be exposed.

Risks of Inconsistency

An IS may rely too much on homemade frontend applications (rich client, web applications) to ensure the system’s consistency. For example, say that when placing an order to the US branch the CSV data has to be posted on two different URLs in a certain order within a certain timeframe. If not, the order’s data will be inconsistent and therefore impossible to process. It wouldn’t be safe to let an external API consumer handle this.

Weak Components

Some existing IS components likely won’t accept a higher level of solicitation. A misuse or a sudden celebrity of the API may result in its breakdown or worse — a complete IS breakdown paralyzing the entire company.

Reason #3 Existing Information System is Slow to Evolve

Those who do not move, do not notice their chains.

Rosa Luxemburg

Because of it’s complexity at the heart of the company, an existing IS cannot easily be modified and is therefore slow to evolve. Since it’s a key process involving organizational, technical and functional matters throughout the company, if our generic eCommerce business wanted to simplify the ordering process it could take months or even years to correct all components and concerns.

How to Transform an Existing IS into a Good API

REST in the front, SOAP in the back. Just watch out for the haircut fashion police…

The API Mullet, Mark O’Neil

A company shouldn’t expose it’s existing IS as is, but that does not mean an existing IS cannot be used as an engine to power APIs. This engine will be hidden behind a new layer: the API layer. The main objectives of the new API layer are to:

• Simplify IS
• Secure IS
• Accelerate IS

The API Layer Simplifies IS

If you can’t explain it to a six year old, you don’t understand it yourself.

Albert Einstein

An API must not be a reflection of the company’s IS; an API must be a representation of its business from an end user point of view. This representation must simplify and uniformize IS interfaces from both technical and functional point of views.

With usability in mind, we can replace our painful ordering cinematic with something simpler like POST /orders. We can construct an understandable interface without unnecessary considerations regarding organizational, functional or technical internal matters. The API layer will handle all transformation and orchestration matters to offer a simplified interface.

The API Layer Secures IS

When exposing an API, a company cannot rely on consumers to secure IS. The API layer is the last man standing to protect the company’s existing IS and therefore must be a safety barrier in front it. The API layer will have to secure:

• Access: The API layer will have to implement an IAM system from scratch or based off an existing one depending on the existing IS security handling. This IAM system will have to control which consumer users which API, which user can access which data, and more.
• Data consistency: If the existing IS does not ensure data consistency, the API layer will have to do it. The complex order placing earlier in this article involved calling two different URLs within a certain timeframe to avoid data inconsistency — this sort of complexity must be handled by the API layer.
• Infrastructures: The API layer secures infrastructures. Some of the existing IS components need help to deliver performance expectations and/or protection to ensure they will not break. The API layer can protect or enhance existing infrastructures by adding some capabilities like caching or throttling.

The API Layer Accelerates IS

The API layer must give the ability to both quickly expose APIs and make them evolve. It must compose with existing IS components or totally new ones without being held back by pre-existing IS lack of evolvability.

Building the API Layer

To successfully transform an existing IS into an API, the API layer must help a company to, at least, easily handle IAM, services transformation and composition, throttling, and caching. But how do you implement these functionalities? Given the needs of the API layer, API management is the perfect solution to expose an existing IS as an API.

API management

Creating a centralized API architecture that makes the process of composing, securing and managing high-performance interfaces significantly simpler and more consistent.

API management definition, the API academy

API management is composed of two parts:

• A gateway handling:
  • Security
  • Transformation
  • Composition
  • Throttling
  • Caching
• A portal offering:
  • Documentation
  • Developer enrolment
  • Analytics
  • Monetization

Choosing Solution(s)

There are many open source and commercial solutions that can fullfil all or individual parts of your API management architecture needs. When choosing solutions to build the API layer, a company will have to consider performance and scalability with SaaS vs on premise computing. Whatever the solution, when building an API layer never lose focus: exposing API must be quick and simple.

Looking Beyond the Existing IS

The API layer is not the panacea — it cannot transform a Peel P50 into a Bugatti Veyron. The caching and throttling capabilities of the API layer may sometimes not be enough to result in an API offering performance and scalability to meet all expectations.

But having an existing IS does not mean that a company always has to use it. Sometimes, to handle really weak components or respond to new needs, the best way to deal with an existing IS will be to circumvent it with new solutions that coexist with a company’s existing IS within the API layer through the gateway.

There’s an API or SaaS Solution

Rather than building custom solutions, there are many solutions that can help companies provide APIs easily. Services such as Restlet API Spark allows one to create an API from a Google Sheet, or Context.io enables one to create an API from emails. As a last resort, it may be time to build applications with an entire new stack. There are now many frameworks that ease the creation of applications with APIs, such as Play framework, a Java or Scala framework, or Loopback, a Node.js framework.

Conclusion: Don’t Let Your Existing IS Hold You Back

Not only do we believe that every company should consider providing APIs, we believe that every company can provide APIs. However, integrating an API with an existing IS does come with challenges. As we’ve said, directly exposing an IS could lead to an API that:

• Is unusable by non-internal IS specialists due to its complexity
• Is dangerous because of its lack of security
• Does not match expected time to market because of its slowness to evolve

To avoid these concerns:

• Don’t let your API expose your IS as is.
• Minimize superfluous complexity
• Use an API layer to simplify, secure, and accelerate IS
• Consider API management and third-party solutions