You may have heard it repeatedly that “API sprawl is the new shadow IT.” But what does that actually mean? Where is this problem coming from? What does this practically mean in the age of AI? And more importantly, how pervasive is this problem across the API industry?

Today, we’re going to look at the problem of API sprawl and shadow IT from a bird’s-eye view. We’ll look at the dilemma of API security, identify how this translates for most providers, and what this means for organizations working with APIs.

What Even Is Shadow IT?

Before we discuss API sprawl, let’s define what we mean by shadow IT. Broadly speaking, shadow IT refers to the presence and use of systems, software, devices, and so forth without the approval — or often even the knowledge — of an organization’s central IT department. This can happen for a wide variety of reasons, both legitimate and illegitimate.

Imagine you are an employee working in a large enterprise in a direct customer support capacity. The consumer requests details about the mobile build of your core application, and to get the best answer, you decide to log in to your development account via the iOS application. Here, you collect the data you need, and you decide to share these screenshots with the user for better support. You upload these images to your Google Drive and share them directly with the consumer using the copy you’ve generated in ChatGPT.

For the average employee, this might seem like a highly optimized flow from initial contact to fulfillment, but for the average corporation, this is shadow IT at its worst. The use of a mobile device without clearance, the use of software without approval, the use of a personal Google Drive to share images… all of this reeks of shadow IT.

Why this occurs is clear: employees will often use the best software or hardware they can to accomplish a task. Their desire for efficiency, the need to bypass a slow or inefficient IT approval process, and their perceived lack of easy solutions for these issues mean that if they can use a better method, they will.

What Is API Sprawl?

Shadow IT is relatively common and easy to define. But what about API sprawl? API sprawl is the uncontrolled proliferation of APIs across an organization. When APIs are created and used without any clear oversight, documentation, or limitations, you get a sprawling network of interconnected or isolated systems that exist outside of the vision — or unified strategy — of the larger organization.

This can introduce significant risks. API sprawl creates some common faults:

• Poorly documented APIs might connect to systems without clear cause or controls.
• Lack of monitoring across APIs that interface with critical systems or serve private data.
• Duplicate or redundant data access might increase overall network and data traffic.
• Orphaned or outdated systems that represent sleeping threats to the stability of the overall network.
• Insecure endpoints that might allow attackers to circumvent controls.

Similar to shadow IT, slow approvals may drive the creation of bespoke single-service or single-purpose API endpoints for specific teams. In other cases, teams create decentralized solutions for more rapid response or individual use. Or, the need for rapid iteration results in new endpoints that duplicate existing ones.

While there’s quite a bit of crossover with shadow IT, there’s also some unique causes of API sprawl that set it apart from other issues of its kind:

• Mergers and acquisitions can add new APIs and platforms that may never actually get unified into the core product, resulting in sprawl.
• Microservices development results in an increased number of endpoints, and when poorly controlled, this can multiply the impacts of poor management wildly in short order.
• Ingesting open-source APIs as “official not-official” resources.

Data Depicts APIs as the New Shadow IT

Recent reports substantiate that APIs are an increasing concern. According to Traceable’s 2023 State of API Security Report, 48% of organizations consider API sprawl to be a leading concern when it comes to API security. That same report also found that 39% of organizations struggled to maintain an accurate inventory of their APIs, and 30% faced difficulties with managing third-party API access.

Postman, meanwhile, noted that 31% of respondents to its study targeting software organizations noted that “managing too many APIs or microservices” was a significant obstacle. This number was higher than the 23% reported overall.

Data suggests this API growth isn’t slowing down any time soon. An F5-backed analysis projects that active APIs could reach more than one billion by 2031.

Unfortunately, all this points to one thing: more and more APIs are being created without controls in place, and those APIs are then being used without much documentation. This is all complicated by the fact that SaaS services like OpenAI provide easy-to-integrate APIs, and their use might not be apparent unless you’re directly looking for it.

Mitigating the Impact of API Sprawl

Now that we understand the problem, the question is: how do we mitigate this risk at scale? While this is still a problem we’re grappling with as an industry, there are some common best practices that organizations can apply to help mitigate the impact of API sprawl.

Establish a Central API Catalog

You can’t manage what you can’t see — and part of where this problem is coming from is the inadequate methods used to catalog and control APIs in the wild. Accordingly, build a centralized API catalog that tracks all public, private, internal, and third-party APIs. This catalog should include the ownership and contact info for all APIs, versioning and lifecycle metadata, security and auth requirements, and any schema or OpenAPI definitions.

This catalog doesn’t only need to be used for monitoring — you could build out support for developer onboarding, internal portals, gateway management, discoverability, and more. Ensure that the centralized catalog becomes a source of truth, not just a seldom-used documentation page.

Shift API Governance Left

Shifting left means moving away from the production release and earlier in the software development lifecycle. The idea here is that governance should be part of the development workflow, and not an afterthought.

You can do this in a variety of ways:

• Introduce linting, schema validation, design review gates, and sunsetting and deprecation strategies into your CI/CD pipelines.
• Review your catalog often.
• Leverage tools like Spectral or Redocly for OpenAPI linting, and Swagger or Optic to detect breaking changes at scale.
• Leverage GitHub actions to block merges on unapproved APIs.

There’s a whole host of things you can do here, but the goal isn’t to slow teams down — it’s to embed visibility and control into the production pathway before you release an API into the wild.

Mandate API Registration

Another excellent method for controlling API sprawl is to use API gateways or service meshes as a control methodology. For instance, you can route all API calls through a gateway, necessarily creating a limitation that sees any API pass through a central point. From here, you can catalog and control the flow of data and APIs into and out of the overall system. You can also implement centralized auth, rate limiting, monitoring, logging, and a whole host of other capabilities.

Notably, this also gives you more remote control over deprecation, version and policy enforcement, and revocation. At this point, you have a system that can only be used by APIs you approve — as opposed to a system where APIs of any stripe can use your systems.

Automate Discovery

Even with these systems in place, you may already have APIs that lack oversight. To mitigate this issue, you can automate the discovery of rogue or orphaned APIs through the use of deep inspection. Leverage network traffic inspection, gateway logs, API firewall telemetry, and other systems to create a record of all API interactions and systems.

This will not only help you with observability, but it will also surface a wide range of data to help identify undocumented APIs, orphaned or zombie endpoints, and even insecure third-party calls. You can use third-party tools like Traceable, Noname Security, or Salt Security for simple plug-and-play implementations, but you can also find a wide range of open source and homegrown solutions leveraging solutions like Prometheus and Grafana to build out native observability stacks.

Enforce API Lifecycle Policies

Finally — and perhaps most importantly — enforce your API lifecycle policies. Every API should have a defined lifecycle: creation, usage, maintenance, and sunsetting or deprecation. Without this policy in place, you end up with forgotten endpoints still accepting traffic, and without enforcement, you have some good-sounding rules without applicability.

To get this right, set some organizational-wide policies, including:

• Versioning, specifically around version formatting, deprecation or sunsetting notices, and other end-user communications.
• Timelines for sunsetting and deprecation, as well as common trigger points for traffic- or volume-based decision-making around end-of-life provisions.
• Expectations and policies for backwards compatibility.
• Procedures for breaking changes, including how these are communicated and how much time users have to transition to new endpoints or versions.

These should be treated as core to the product itself. Beyond just being good to document, they should be the guiding principle behind the lifecycle of the product offering at all stages.

Mitigating Shadow APIs in the Agentic Future

This is not a solved problem. API sprawl is rapidly growing, and with the advent of large language models (LLMs) leveraging Model Context Protocol (MCP) and APIs, the problem is likely only going to grow exponentially. In the next three to five years, we will need to come to terms with more effective controls for agentic access, and with this, we will need to get better at cataloging and controlling our APIs. The work we do now will set the stage for that era — so, investing now to get this right is investing in a future where API sprawl is far more controlled.