API Lifecycle Retirement Stage: A History of Major Public API Retirements Bill Doerrfeld April 23, 2015 The API lifecycle is an agile process of managing the life of an API. It’s made up of 4 distinct phases: Analysis, Development, Operations, and Retirement. Old age comes, and retirement is a part of life. It’s no different in the API world. The retirement of the Netflix, Google Earth, and ESPN APIs are examples of some recent large public API deprecations. Every major API retirement tends to spawn worry within the developer community; are public APIs doomed? We don’t think so — don’t forget there’s currently an estimated 13,000 web APIs in operation, and the sector has shown exponential growth and is slated to increase with the IoT (which isn’t just a fad). However, all things come to an end. As conditions change, technologies must evolve to meet new market conditions. A public-facing API may prove successful for some organizations, whereas for others, open environments may not be viable and lucrative. What Does Retirement Mean? Retirement can mean many things. An API version may be scheduled for complete deprecation, or the API could still exist in a limited format with increased access controls. An API may simply slim down it’s excess functionality to become more consolidated and lean, or made obsolete by an internal technology or outside competitor. In order to fully understand the conditions that beckon major API changes, we’ll lay out 5 common causes that beckon API retirements. Using large API deprecations from recent years as examples, we’ll explore the reasoning behind each one that we’ve been able to surmise through our research. Sound fun? We think it’s pretty fascinating what can be learned from hindsight… Retirement Reason #1: Lack of 3rd Party Developer Innovation A common scenario that may cause an API shutdown is if the API is receiving limited or unsolicited use by developers and end users. Though this could be due to inadequate marketing, or it could be proof that the API is not offering a true value to it’s consumers. Netflix Netflix proudly unleashed their API program in 2009, citing the possibilities for new app creations by third-party developers. Eventually they stopped issuing new API keys to developers, and by 2014 this was shut down to all users aside from select partner integrations. According to Daniel Jacobson, Director of Engineering at Netflix, their reasoning was “to better focus our efforts and to align them with the needs of our global member base.” In the end, all of Netflix’s public API requests equated to about one day of private API requests. The user base for the public API was too miniscule to warrant it’s existence. The result was to eliminate their Public API program to embrace a single private/partner Java API. Nowadays, every Netflix app on any device taps into the Netflix private API, which performs data gathering, data formatting, and data delivery for all consumers. Netflix has also eliminated versioning within their API to embrace impermanence and constant change. We learn from this case that when a brand name and native platform is so powerful, creating an ecosystem with a public perhaps not necessary. Also, according to technologist Andy Thurai, it “becomes very expensive and time-consuming to maintain” both a Public and a Private/Partner API. Retirement Reason #2: Opposing Financial Incentive, Competition A Public API may be retired if the interface interferes with primary business objectives. In exposing their data for anyone to consume, providers run the risk of losing a market advantage, especially if this is taking business away from native distribution channels. ESPN In the case of ESPN, their reasoning to retire their sports data platform was “to better align engineering resources with the growing demand to develop core ESPN products on our API platform.” Today, if you visit the ESPN Developer Center, you’ll notice that all APIs are open for “strategic partners” like IFTT, Pulse, and foursquare. Discussion on Hacker News suggested that it was because developers were monetizing their apps. Andy Thurai agrees, positing that “the actual reason seems to be that they want better control over their unique and licensed content, and better monetization, which public APIs may not offer.” We learn that by exposing internal assets, a business may risk losing an advantage if doing so decreases traffic from native distribution channels. Strava Strava offers fitness software for wearables with a developer API to extend their applications. Starting with Strava API v3, they began to curate their developer consumers with a limited access program. These new requirements prohibit “applications that encourage competition” as well as ones that “[Replicate] Strava functionality.” A large API retirement can occur when developer consumers are incorporating monetization techniques into third-party apps that break the original terms of agreement. For more details, read the v1 & v2 Retirement Announcement. Retirement Reason #3: Changes in Technology & Consolidating Internal Services The tech sphere is constantly evolving. As such, advancements in these technologies have the potential to make an API obsolete. This happens often during internal redesigns, acquisitions, or external industry advancements. API retirement could arise from evolving protocol trends, such as REST replacing SOAP, or changes in end user experiences, such as Netflix shifting it’s primary business model from DVD distribution to video streaming. Skype Skype’s reason to decommission their Desktop API was, according to them, that it didn’t support new mobile development trends. The Skype Public API, first introduced in 2004, was taken over by Microsoft in the 2011 aquisition. The Microsoft-owned Skype Desktop API was shut down in order to embrace Skype URIs and the Microsoft client. The termination irritated some developers as it erased income for third-party applications consuming the API as well as lessened functionality through the Skype URIs. For more details, read the Retirement Announcement. Google Maps Data API At the same rate Google releases new APIs, it also deprecates older versions. In 2010 Google announced they would discontinue the Google Maps Data API, a service made to store geospatial data. The main reason was that a superior feature was launched with v3 of the Google Maps API, offering an improved method of query and storage that made the Google Maps Data API obsolete. For more details read the Retirement Announcement. Some other Google APIs scheduled for deprecation due to outdated technology: Google Webmaster Tools Google SOAP Search API Fedex At the time of this writing, Fedex is in the process of updating its APIs to Fedex Web Services, replacing XML-based tracking applications with a large redesign. For more details, read the Retirement Announcement. Retirement Reason #4: Versioning Versioning is by far most common reason to retire an API. Since APIs are a relatively new technology for most businesses, you’ll often see v1, v2, or v3, still in use today. Similar to OS upgrades on devices, it’s often easier to scrap and replace an entire version when instigating large change. Versioning can arise when major edits to the API parameters or new usage controls are required. Some alarming changes that dramatically affect developers and end users may be quietly hidden within new API version documentation. Functionalities may be erased, permissions limited, classes altered, etc. As versioning is a common component to many API’s lifecycles, we don’t intend to create an exhaustive list of them all. Rather, these two examples of version updates came with them a few surprises for developers and end users… YouTube YouTube, in the process of migrating to the YouTube Data API v3, surprised users when it was found that Data v3 would no longer support the YouTube app on expensive Smart TVs. Twitter v1 Twitter’s v1 was finally retired in 2013. Being the 3rd most popular API in use today, the announcement sparked worry throughout the developer community that simplistic integration was a thing of the past. The update added OAuth, requiring client keys and the creation of an app with Twitter. According to Mathew Mombra, these changes were likely instigated to: enforce call rate limiting with less liberal access reduce their system load decrease API vulnerability promote their embeddable tweet program Retirement Reason #5: Security Concern A consideration of many large-scale retirements, API providers may choose to discontinue a public API specifically because of the security concerns associated with making internal data and processes public. Google Earth API The API was shutdown because it relied on NPAPI Framework, an outdated technology Google mentioned had become a security concern. SnapChat Though SnapChat never technically has released a public API, the popular app has spawned a variety of mashups on various platforms that all tap into their “private” API. This went largely unchecked, until early 2015 when SnapChat partnered with the Windows store to crack down on these rogue applications. Anticipating Deprecation: What Happens When I Change My API Parameters? Though most clients are quick to adapt, some may be unaware of API changes. This means they won’t respond quickly to update their system to be compatible with new versions. Stagnancy, freezed dependencies, forgotten forks, and inactive projects are common with APIs that have been active for a while. Mircea F. Lungu, a computer science researcher at Bern in Switzerland, found that when deprecating API methods or classes, the total adaptation time within an ecosystem can vary tremendously. The amount of time between the first reaction and the last reaction can be months, or even years. In his study on the case of the SmallTalk ecosystem of 2,000 consumers, Lungu found that 14% of deprecated methods induced negative rippling effects within the ecosystem, and 7% of deprecated classes triggered ripple effects. Only 16% of deprecations had a systematic replacement, which meant that most edits were done manually. Thus, any changes to an API induces ripples that can have far-reaching results. Preparing For Developer Reaction Entire software ecosystems can quickly emerge that depend upon APIs to survive. And just as quickly, they can be destroyed. As these third-party developers have depended on your service to support their own products, complete shutdowns can inherently cause very negative reactions, especially if not executed well. In order to keep good face with your community, we recommended transparency using the following techniques to prepare for an API’s version retirement or complete deprecation. Schedule a retirement plan: Offer at least a 6 month time frame for complete deprecation. This will hopefully give teams sufficient time to seek new integrations or partnerships. Consider extending this time frame if certain teams are struggling. Google Maps and Youtube follow a one year announcement to deprecation policy. Make changes in stages: Tier your retirement plan in a way that aligns with past promises for support and versioning. If still running, shut down older versions that receive limited use earlier in the process. Twitter has created handy retirement timelines in the past that display when each version will be retired. Public announcements: Write and promote a blog post that explicitly outlines API changes, when they will go into effect, who will be effected, and any other vital information your community needs to know. Disseminate this information to your dedicated developer social accounts. Ease the transition: If your API is being consolidated, versioned, or merged into a separate service, ease the process with a migration guide and FAQ. For example, when Google Maps Data transferred over to Google Maps, they created a Maps Data Liberation Tool to offer a complete download of data or transfer to their new console. Awkward transitions can especially be a headache during acquisitions. Use blackout testing: Blackout testing is a planned shutdown of all API functions for a short window of time. This is helpful for developers to see how the absence of the API is going to affect their systems, and also acts as a call to action for developers to update their code. The Youtube API, Twitter API, Mendely API, and many others have used blackout testing in the past. Don’t violate your own terms of service: Review your Terms of Service to ensure that you have no binding contracts for continuing support before you pull the plug. A smart way to plan ahead is to write a deprecation policy whenever a new API version is created. Include a timestamp in the Sunset HTTP response header field: This will communicate that a resource is expected to become unavailable in the near future. See this IETF specification authored by Erik Wilde for more information. Next Steps: We’ve highlighted the major concerns that would usher a large-scale API deprecation, but many retirements occur for a combination of many these reasons. Scheduling the retirement of an API version may be a great advance toward restructuring how your data will be received by your consumers. In this case, revisit the Analysis Stage to consider your primary business objectives. Other reasons to consider large-scale API depreciation or redesign may include: Overwhelming competition with other APIs in niche sector Inability to structure data to be received effectively Not generating a revenue to support operations with current offering and many others Choose Your Adventure: Advance to any of the following articles in the API Lifecycle series: Introduction: Envisioning the API Lifecycle Analysis Stage: Preparing your API Strategy Development Stage: Deploying Your API Operations Stage: Marketing Your API Retirement Stage: A History of Major Public API Retirements — The Ultimate Guide The latest API insights straight to your inbox