Shadow APIs present many potential vulnerabilities for a modern organization. Undocumented endpoints, abandoned APIs, and unofficially-supported functions… all of these items constitute shadow APIs and can create serious headaches for a cybersecurity posture.

Below, we’ll look at some simple tips to ensure that shadow APIs remain a manageable problem in the API space and consider some ways to avoid shadow APIs altogether.

Proactive Measures To Take

Much of the issue around shadow APIs must be resolved proactively. For developers creating new solutions, keeping this threat in mind from the onset can pay serious long-term dividends regarding security, brand confidence, and general efficacy. With that in mind, here are some proactive measures to avoid the shadow API problem.

Document Everything

Solving this problem begins with one step that developers should already be implementing — document everything. With each new thing developed in an API, the complexity of the interconnected system grows. This complexity grows even further as these new developments are connected to other endpoints and data systems. Documentation is an important step in any API lifecycle process, as it outlines the knowledge of functionality for future use.

Many shadow API issues exist not because something has been purposefully abandoned or forgotten but due to developer turnover. It’s quite possible that something developed ten years ago in an API was not created by anyone currently working in the organization. For this reason, documentation is a significant factor to adopt early and as a core process.

Implement Automated Solutions

The best system to prevent shadow API development is one you don’t have to think about. Any manual process will be prone to human error, so adopting automation can help minimize this risk.

This automation can happen at several levels, but the most obvious (and the easiest to implement) would be in the API documentation. Utilizing options such as Swagger to automatically generate documentation based upon the endpoints fed into the system will allow for a greater level of transparency. With greater transparency and documentation, it becomes much harder to exist in the shadows of an API system. This can go far to ensure shadow APIs are managed and minimized.

Understand External Dependencies

In some cases, exposure from a shadow API can actually come from outside the API system itself in the form of external dependencies. Utilizing external APIs is often necessary, but in doing so, you put a lot of trust into a system you have very little control over. These APIs are just as fragile and potentially vulnerable as your API is, but with the added layer of inaccessibility to find and fix shadow constructs.

In these cases, two approaches complement each other. The first is simply being proactive with each solution. Implementing things like stringent access control and authentication can help mitigate threats if they might occur on outside dependencies. Encrypting at rest and in transit and ensuring that all sensitive data is handled via first-party or secure modalities can help further mitigate these threats.

The second approach is to ensure that consumers are subject matter experts for the third-party APIs in question. This is a daunting task, to be sure, but if development routinely audits those APIs as if they were internal, much of the concern around external dependencies can be mitigated.

Keep Testing to Testing

Whenever possible, ensure that your endpoint testing is done in an actual testing environment, not in a production one. For novice developers and small-scale API providers, it is not uncommon for new endpoints and tests to happen via a singular service or API. In many cases, there is no testbed as much as there is “deploy and see if it works”. This is a problem for many reasons, but the most obvious one is that deploying systems that are being tested opens a massive vulnerability that may be difficult to close down. This simple fact is that developers can’t really know where their API will ultimately end up — a fun side project API could very well be an enterprise-level tool in a handful of years. Simple, early mistakes like this can propagate into future versions quite easily.

Part of the danger here lies in how such endpoints are used. When they are used for rapid testing, they tend to be temporary tests. The exposure can be potentially damaging if these temporary endpoints are then released into the wild.

Remove Endpoints When Necessary

Many shadow API issues revolve around the reluctance to remove endpoints, especially when legacy users still utilize them. The reality is that there are many ways around this issue, and simply providing the old functionality without sunsetting and deprecating the endpoint only creates additional problems.

However the API manages to implement new endpoints, ensure that an endpoint that is no longer supported is actually removed. In the quest to support as many users as possible, you should not be introducing any complexity or documentation failures. It’s better for the userbase, in the long run, to lose out on a feature than it is to create abandoned endpoints and shadow APIs that lessen the overall security of the platform.

How to Address Shadow APIs When You Discover Them

Unfortunately, not all developers will be lucky enough to implement solutions to shadow API issues ahead of time. You may very well have a hunch they exist, however. Here are some tips on how to discover them and address them once found.

Audit and Audit Often

API auditing is the process of testing an API to ensure that it matches the described form and function against an expected result. This seems simple, but years of scope creep and add-on functionality can result in an API that looks starkly different than even the lead developer may realize.

Accordingly, developers should audit and audit often. Auditing should become a core part of the development lifecycle: audit before major releases and audit in between new feature tests. Auditing should become a way of life as a process to ensure that the build output matches the expected form and function.

Note that auditing can take several different forms. Routine auditing should be done to check against the development design and plan, especially when multiple teams are involved. But every so often, deeper audits are likely required. Testing the system for any unexpected behavior, attempting to override security features, and other such security audits should be engaged semi-frequently to ensure that security postures are maintained. Forensic audits should also be conducted to ensure that traffic behaves as expected. This should complement a heuristics system to lock down any unexpected behavior, but these kinds of audits are also very effective methods to discover overfetching and poor pagination.

Backport and Update Existing APIs

In some cases, shadow APIs are not necessarily abandoned — it may just be that their use was unknown. Functions that are still useful may be resolved by older APIs that have not been updated because their use was poorly documented or understood. In such cases, there are two options for resolving this issue.

The first is to simply backport the APIs. In backporting, security features, updates, and revisions are pushed to out-of-date or deprecated APIs that are still used for whatever reason. In these cases, it may be worth updating the older APIs to align with the core API code, especially if building their functionalities into the core API is underway but not yet fully at a deployable state.

That being said, the best approach is the second: simply update the existing API to have the functionality it depends on the shadow API for. If your core API is referencing a deprecated and out-of-date API, but still utilizes that API to perform critical functions, then the function should be added to the core API base. Or, it could be spun out into its own, new microservice that is updated and well-documented.

Implement a Development Control Process

Once shadow APIs are discovered, processes should be put in place to ensure that the issue does not repeat itself in the future. In its entirety, this should take the form of a control process, preferably embedded into the development lifecycle itself, and should focus specifically on preventing shadow APIs through periodic reviews, consistent policies, and forward-looking design paradigms.

Review Existing Codebases Regularly

As part of an ongoing development process, APIs should be regularly reviewed as part of the development lifecycle. When a new feature is added or a new dependency introduced, the base it touches should be looked at, and its endpoints should be polled and matched against existing documentation. One way to think of this process is pruning — as a good arborist would prune a tree to help it stand tall and strong, so should a good developer regularly prune any deprecated endpoints, old code, or abandoned functions.

The reality is that it will be almost impossible to eliminate any abandonment issues or deprecation. The nature of development is such that new systems constantly replace the old, and some components will inevitably be left behind rather than entirely removed. Making reviews an ongoing process will create another safety net that will catch human errors before they propagate into larger problems.

Establish a New Implementation Review Process

As part of the process moving forward, any dependency should be heavily reviewed for its own shadow APIs, and moreso, should be treated as if they will eventually deprecate and result in internal shadow API issues. Most developers have adopted a zero-trust model in terms of security, and in many ways, this kind of zero-trust thinking should also be applied to external APIs.

Regularly check against published documentation to see if your materials still match the public-facing documentation and API behavior. Review ongoing data transfers, and track deviations that suggest undocumented changes or variations. Track the changelogs of external APIs, and ensure that, if an update is required for the API or dependency to properly be considered in the mainline development branch, the API is using the updated code, modalities, or systems. In essence, if you have to use a dependency, treat it as if it will be deprecated tomorrow and plan for that eventuality. This will make any shadow API issues arising from abandonment or depreciation easier to recover from.

Design for the Future

When developing new APIs, designing for the future can help avoid creating shadow IT altogether. As you develop, consider how the endpoints relate and how they will be supported long-term. Too often, developers think that a simple endpoint creation is no big deal, but when you adopt this mindset, you can easily over-create, causing bloat and eventual abandonment.

That’s not to say that API development should be artificially limited —more specifically, API development must balance what is useful in the intermediate and long-term. Avoiding flippant creation and constantly asking “is this necessary” can help optimize scope and avoid service abandonment.

Conclusion

Shadow APIs can be problematic, but with a bit of forethought and planning, their threats can largely be mitigated. A small amount of organizational planning can set up API developers for a more efficient and well-structured future. What do you think of these tips? Let us know in the comments below!