APIs are continually transforming our digital world by connecting software, data, and processes with people. Within the last decade, many technological innovations have come together to help enterprises with different aspects of API management.

API management means something different within every organization. It can include API design, specification, documentation, prototyping, versioning, testing, analytics, monetization, lifecycle management, API discovery, and similar functions. These areas are evolving as enterprises adopt various tools and processes to manage their APIs better. However, API discovery is one such topic that, unfortunately, has evolved to become more ambiguous than before.

API discovery has attracted more attention from the developer communities in recent times, partially because of the growth of APIs across various industries. With this growth of APIs, I mean both ways, i.e., building APIs to be consumed by others as well as consuming APIs built by others. Searching for APIs and testing them before using them in live products and services has become cumbersome due to the sheer large number of APIs that are available publicly.

The API Population Dilemma

At large enterprises, publicly available APIs could easily run into the hundreds. Consider, for example, Google and Microsoft, which currently have 183 and 132 APIs, respectively, listed in their respective API marketplaces. Without the ability to explore APIs easily, it will be difficult for developers to use their APIs seamlessly. Arguably, exploring Google or Microsoft APIs could take days, if not months, for any productive usage or application integration. Imagine exploring APIs from tens or even hundreds of vendors.

With the current growth of APIs, it is conceivable that, in the near future, developers may need to search through millions of APIs to find the specific tools that meet their business and technical requirements. This usability gap is the crux of the API discovery problem.

Timeline of Defining API Discovery

Before we explore what API discovery means in today’s fast-evolving digital economy, let’s review a brief history of API discovery, as defined by industry professionals and academia.

• 2014: In academia, Jayathilaka, Krintz, and Wolski defined API discovery as: “the process of identifying available and appropriate Web services by clients, which is facilitated through automation” in their book titled: Service-Driven Computing with APIs: Concepts, Frameworks, and Emerging Trends.
• 2014: Steven Willmott from 3Scale (acquired by Red Hat) defined API discovery as “functionality for finding APIs.” Furthermore, he categorizes two primary methods of API Discovery: directory route and search route. Essentially, what he meant by the “directory route” is API marketplaces where APIs can be cataloged.
• 2015: Martin Buhr from Tyk.io provided two basic ways to achieve API discovery through Developer Portals and Machines. Developer Portals that provide: (i) documentation, (ii) the ability to experiment with API, (iii) the ability to sign-up to use APIs, (iv) enrollment into API access tiers. Machines are supposed to discover, interpret, and engage with APIs automatically. Note that Martin inherently assumed that the API Discovery is the process of finding APIs manually or through machines.
• 2015 Bill Doerrfeld surveyed various API marketplaces, directories, and lists. He defines API discovery as “a process of collecting and cataloging APIs.” He emphasized in the article that the scalability of this process is weak, considering the future growth of APIs in the hundreds of thousands or even millions of APIs, and the lack of standard, machine-readable definitions. He then goes on to present 15 ways (marketplaces or directories) to find APIs.
• 2018: Kin Lane described API discovery as an “ongoing operation of API management.” He explained that this operation begins with the need to find all digital resources within an enterprise. This requires reaching out to every group within an enterprise to ask for various information on their APIs and then aggregating all documentation and other details on APIs to make it useful for developers.
• 2018: Broadcom (provider of CA API Management platform) described API Discovery as an enabling process that increases development efficiency and innovation. For instance, if API developers are creating services that provide access to applications or model digital business capabilities, it is crucial to make capabilities discoverable and easy to use. To achieve that goal, an API catalog can be provided for developers to explore APIs with interactive documentation and testing tools while also providing registration and application management capabilities (that are going to consume the published APIs). In summary, Broadcom’s API discovery definition is about providing tools and processes to enable the exploration and consumption of APIs.
• 2018: Diego F. Serrano Suarez studied the automated API discovery problem in his Ph.D. thesis. He highlighted that developing applications that rely on web APIs still requires a considerable effort from developers, who need to find relevant APIs, write code for invoking specific APIs, and transform the data to confirm with their application structure. Diego inherently acknowledges that “discovery of API” is a challenge for developers.
• 2019: Arguably, Kin Lane wrote the most up to date commentary on API Discovery in an interesting article published on DZone entitled “The Complexity of API Discovery.” Instead of defining API discovery, he considered different aspects of API discovery from various roles such as “API Provider,” “API Consumer,” “Analyst,” “Investor,” “Journalist,” “Academic,” “Government,” and “End-User.” It is worth noting that “End-User” here does not necessarily mean the developer involved in building or providing or consuming APIs, but an “End-User” here actually means a person that is using the software applications (mobile, desktop or enterprise, social) in their daily life. Kin has made an important observation, and I will come back to that in the latter part of this article.
• 2020: More recently, RapidAPI explained API discovery as the process of finding APIs. Like Bill Doerrfeld, they surveyed various API marketplaces and directories.
• 2020: The most recent yet very product-specific definition of API discovery came from SmartBear, which defined it as the process of recording requests to a web service to create a test case or an API definition.

It is evident from the above discussion that API discovery is generally defined as the process of finding APIs from API marketplaces, directories, or through search engines. Unfortunately, this definition of API discovery is not comprehensive enough for today’s API economy. The process of finding APIs will continue to be the soul of API discovery, but the practice has become more complex. Other aspects could broaden the scope of API discovery, which eventually may help in addressing the challenges of API management.

The discovery of APIs is typically viewed from the perspective of API consumers. However, the discoverability of APIs is equally essential to API producers as it could dramatically affect their marketing efforts.

Also, note that API producers and consumers may represent various groups within an enterprise. Development, security, operation, legal, licensing, and business teams could all be interested in APIs. The point is, in today’s complex enterprise environments, organizations are usually collectively responsible for managing APIs from a consumer as well as producer perspectives.

5 API Discovery Pain Points

I do not intend to provide new definitions or propose new tools or processes. Rather I am going to highlight some of the key pain points that enterprises experience (or will experience shortly) due to the increased usage of APIs across various industries. By broadening the scope of API discovery with these features, we can naturally address some of these pain points. Some of these highlights are:

1. Technical Due Diligence

Legal (or compliance) teams at enterprises struggle with approving “freely” available APIs. Free does not necessarily mean “no obligations.” Therefore, as part of the technical due diligence, developers need to work with legal teams to meet compliance requirements before integrating such APIs with enterprise products.

One of their challenges is to track the usage of such APIs across various products. The number of “free” API calls may be limited, and technical use cases may be restrictive. API terms of use may change, the API hosting servers could be in a jurisdiction that is not compliant with regulatory requirements, and so on. Therefore, when we are talking about API discovery, we really need to go beyond knowing the vendors, API endpoints, and API documentation; we require due diligence that covers technical and compliance realities.

2. Terms of Use Transparency

Security and technical legal compliance are collective responsibilities. At an enterprise (as an API consumer), we can’t expect security engineers to find and manage all the vulnerabilities. Similarly, we shouldn’t expect lawyers to detect and mitigate every single legal risk. Even before developers start to “play” with open (publicly) available APIs, if they can quickly discover the terms of use and their “Dos” and “Don’ts,” they can further strengthen the risk posture of their respective organizations.

3. Security Discoverability

Similarly, discovering the security posture of APIs (especially third-party integrations) is non-trivial. The vendor’s reputation is helpful, but you can’t take things for granted. For example, you may very well trust that APIs from Google and Facebook are well designed and protected before your product starts to depend on them. Still, misconfigured APIs that leak personal data from hundreds of applications could shake that confidence. Furthermore, APIs may be hosted on servers that are vulnerable to attacks because the underlying software is vulnerable. We have seen several examples in the past, including the Apache Struts vulnerability that affected Equifax.

4. Regulation Compliance

Compliance with regulations (such as GDPR) may require enterprises to store data in specific regions. With the integration of third-party APIs (including open and free APIs), it has become more challenging for enterprises to meet their regulatory obligations. Discovering API server locations before such APIs are integrated is vital for enterprises to avoid legal pitfalls.

5. Open-Source API Visibility

Developers often use open-source libraries to accelerate the development process. Often, these open-source libraries also provide various API integrations. Discovering embedded APIs from open-source libraries brings more visibility and helps enterprises strengthen their security and compliance positions.

Final Thoughts

In conclusion, we have witnessed a rich history of dialogue on API discovery. Most of these explanations primarily consider it as a process of finding APIs from marketplaces, directories, or search engines. I have tried to enrich these explanations with new aspects of API discovery. In addition to API documentation and vendor information, API discovery should also consider technical limitations, terms of use, legal compliance, and security concerns. These elements are needed for building a new generation of API management solutions to fuel the growth of the API economy in a more secure and compliant way.