Twitter’s 10 Year Struggle with Developer Relations

In September 2006, only a few months into its existence, Twitter came out with the first version of its public API. This was surprisingly early in an age when social APIs were not yet prevalent, especially since Twitter had yet to become the success story that seems so obvious in hindsight. In fact it was mostly a reaction to third party developers scraping their website Twitter.com and creating unofficial APIs.

It marked the start of a long and at times contentious love-hate relationship between Twitter and third party developers. In this article we delve into some of the events of the last 10 years and explain the motivations behind some of Twitter’s most controversial decisions. We’ll also compare Twitter’s record on developer relations with those of two other social networks – Facebook and Instagram.

2006 – 2010: The early days

The first version of Twitter’s API was a free-for-all and instant hit. It opened up a lot of the social network’s inner workings to developers, enabling full access to tweets and content published by users on Twitter. Any developer could use the API — they simply needed to authenticate using the username/password combination of their regular Twitter account. No further limitations were imposed. The sheer amount of data offered generated plenty of interest among developers, and many companies started building products on top of the Twitter API — apps included Favstar.fm, DailyBooth, TweetDeck, Tweetbot, Echofon and Twitterrific.

Just as Twitter’s user community was credited for inventing some of its defining features, like hashtags and retweets, the API developer ecosystem also paved the way forward. Among the early third party apps, TinyURL and bit.ly enabled URL shortening within tweets, Summize offered up the first full-text search engine on top of Twitter, TwitPic let Twitter users share pictures, and Tweetie — not Twitter — built the first Twitter iPhone client.

Some of the other apps (like Echofon and Tweetbot) were Twitter clients that offered an alternative to Twitter.com for users who wanted to consume the same content through a different user interface (Twitter would later call them ‘traditional Twitter clients’).

2010 – 2012: OAuthcalypse, competing with third party apps and other perceived betrayals

The honeymoon between Twitter and third party developers ended in 2010 when Twitter’s management enacted a series of decisions that were greeted with anger and disappointment.

Twitter announced that all third party applications requesting data on behalf of its users needed to authenticate to the Twitter API using the OAuth protocol. This change was inevitable for security reasons — OAuth was quickly becoming the standard for secure delegated authentication and authorization, and afforded a much better alternative to the Basic Authentication scheme that was in place before its adoption by Twitter. It also afforded the company better visibility over which applications were connecting to the API.

However this change caused distress among the ranks of the developer community, as many apps had depended on their API for years and weren’t ready for such a change. OAuth was not yet widely adopted at the time, and many developers struggled with its implementation, not to mention the changes in user experience it brought along. Bloggers were quick to call it the ‘OAuthcalypse’, and ‘#OAuthcalypse’ became a trending topic on Twitter itself.

While imposing OAuth is easy to defend in hindsight, Twitter also started building features within its own website that competed with or replaced existing third party apps. Starting a trend that it would repeat several times in the future, it purchased Tweetie (renaming it Twitter for iPhone) and Summize (renaming it Twitter Search) and announced that they would become part of the core platform, striking fear in the hearts of developers building competing services.

Twitter developed their own URL shortener and threatened to deny all others. During Twitter’s Chirp developer conference that year, then-CEO Evan Williams pronounced: “We are probably not going to give people a choice. If they want to use a different shortener, they can use a different app”. Once again this was badly received, although Twitter had solid reasons for introducing a native URL shortener at the expense of competing specialists — namely security and analytics.

Finally, Twitter signed partnership deals with Google, Yahoo, and Microsoft to include tweets in the results of their respective search products. Furthermore, it gave exclusive data reseller access to Gnip, a data reseller it would later acquire. Both of these decisions, motivated by Twitter’s thirst for additional traffic and their need to generate revenue, caused consternation when they were announced.

2012 – 2013: Token limits and open war on traditional clients

Beginning in 2012, Twitter further tightened their terms of service. It imposed a 100,000 token limit on connected users for apps that “mimic or reproduce the Twitter consumer experience”, crippling many of them, and set per-endpoint rate limits across the board. OAuth authentication became mandatory for all endpoints, and Twitter introduced Embedded Tweets and Timelines as an alternative to applications that wanted to replicate part of the core Twitter experience in their own apps.

Two main reasons explain these changes. Having raised huge amounts of venture capital, and in preparation for a late 2013 IPO, Twitter needed to increase its ad revenue. As the conflict between its willingness to attract developers and its need for monetization increased, Twitter started to openly discourage the use of the API by those building tools that would compete for eyeballs with Twitter.com. In addition, Twitter had to some extent become victim of its own success and had faced many technical problems and outages. Curtailing the use of its API would help control the amount of traffic that was allowed from third party apps.

In the wake of these changes several Twitter competitors such as App.net were started, but such was Twitter’s dominance of this space that none have come close to dethroning it.

Read our article on the 5 Main Reasons for API Retirement

2013 – Present: Post-IPO controversies

Any hopes that developer relations would significantly improve following Twitter’s IPO would soon be dashed.

In 2015, shortly after live video-streaming startup Meerkat became a hit, Twitter acquired its competitor Periscope and temporarily revoked Meerkat’s access to the Twitter API. It also acquired Gnip and shut down agreements for resale of data with its other partners (DataSift and NTT Data), which was qualified as an “evil move” and “innovation destroying”.

Later the same year the Open State Foundation’s API’s access was suspended. The Foundation had leveraged Twitter’s API to archive deleted tweets by politicians and make them searchable on its website Politwoops. Twitter later restored Politwoops’ API access, but this was another blow to its reputation as an open platform and an enabler of innovation.

Most recently, in November 2015, Twitter removed the Tweet count JSON endpoint to public outcry. The REST API still has the same features, but it is less precise and exposes the data in an aggregated or limited manner. The only way to get the same analytics data is through Gnip’s expensive Enterprise Search API.

Wooing back developers

Despite all of these controversial decisions, Twitter’s developer ecosystem remains healthy due to its unique status as a major social network. Many companies built on top of Twitter’s APIs are alive and doing well today, like Buffer, Hootsuite, SparkCentral (formerly Twitspark) and Storify.

Nevertheless Twitter is facing challenging times, with layoffs, a change at the helm of the company, executive departures, and a declining share price. Twitter’s management understands that they need to invest in a strong developer ecosystem to remain relevant going forward.

In particular, Twitter needs to find new monetization channels, and the chief battlefield is mobile, where Facebook is currently outperforming Twitter, in part due to lack of developer interest in the Twitter platform.

In the last few years Twitter has turned its focus towards mobile developers. It has recently developed or acquired a set of new tools to help developers build better mobile applications while using the Twitter API, and indirectly generating more revenue via their MoPub mobile ad service.

Developers are therefore critical to the next phase in Twitter’s evolution, but its reputation is holding it back. At the Flight developer conference in October 2015, newly reappointed CEO Jack Dorsey offered an apology to developers for past mistakes, echoing the words of his predecessor Evan Williams five years before, and promising to usher in a new era of collaboration between Twitter and its developer community.

New releases and optimism going forward

introducing-fabric-homepageBeyond these soothing words, Twitter showed how determined it was to win back developers by announcing its new capabilities. Many of these revolved around Fabric, its mobile development platform, which now includes the popular mobile crash monitoring and reporting tool Crashlytics, a recent acquisition, along with two key new features: Beta, an app distribution and tracking tool for beta testers, and Answers, a mobile analytics dashboard.

In addition to Crashlytics, Fabric features the Twitter Kit, the rebranded Twitter SDK for mobile development, and Digits, a simplified authentication mechanism based on phone numbers rather than usernames and passwords (an enabler of SMS-based services). Another recent acquisition that made its way into Fabric is Fastlane, a tool that eases the deployment workflow on apps for iOS and Android.

Time will tell whether these efforts will convince developers to give Twitter another shot, and if the tension between the company and its developer community will subside. One reason to be optimistic is that Twitter is consciously steering developers away from building features on top of its core product and is positioning its API as an enabler instead, in the same vein as developer darlings Stripe.

Twitter is often singled out for its problems with the developer community, but its advocates argue that this is an unfair assessment, pointing to Twitter’s numerous contributions to open source software (such as Bootstrap, Bower, FlockDB, Gizzard and Finagle).

That concludes our research into the history of the Twitter developer program. However, other social networks have had similar woes. As an addendum, the next sections briefly look at two other giant social networks and how they handled relations with their respective developer communities.

Other social networks

Developer relations at Facebook

Facebook had the relative luxury of having a stable vision (and a stable website) before public social APIs became fashionable. Launched in 2006 — shortly before Twitter’s API — Facebook’s Developer API gave access to its users’ friends, photos, events and profile information. This ushered in a golden age of social gaming, with game developers like Zynga, the makers of Farmville and Mafia Wars, and PlayFish, the makers of Pet Society and Who Has The Biggest Brain, benefiting from viral effects that only Facebook could enable. In addition to games, applications that leveraged Facebook’s social graph, like Zoosk, got a huge boost from the Facebook platform.

A few years later though, as users complained about spammy apps and games spoiling the user experience, Facebook gradually curtailed the viral marketing tactics that powered these applications, thereby also limiting its own appeal as a development platform.

Facebook had its share of failures with their developer API, and was the recipient of a great deal of criticism from developers. It unsuccessfully tried to launch a virtual currency during the early years, and more recently shut down Parse, their mobile BaaS acquisition beloved by mobile developers, although they allowed the code to be open sourced and gave developers a one year notice period to migrate away.

One area where Facebook has made a mark is in delegated authenticationtheir OAuth API is highly popular due to the sheer number of people with Facebook accounts and the social proof that users can benefit from connecting to an application using their Facebook account.

Developer relations at Instagram

Instagram seems to have benefited from being a late addition to the ranks of super-social networks, with the advantage of hindsight helping them avoid mistakes, but at the same time suffered from lack of resources in the early years.

Instagram was launched in October 2010 and already had a million users after only three months. By that time, it was unthinkable for a major social network not to release its own public API. Because of its astonishing growth, Instagram was caught unawares as developers started clamoring for a fully-featured API.

The company wanted to focus on the core user experience, avoiding distractions, so they didn’t immediately accede to the demands of their would-be API consumers. But because Instagram was a mobile-first product, it was possible in December of the same year for a developer called Mislav Marohni to reverse engineer its private API’s initially unencrypted calls to and from the mobile UI and create an unofficial API. At least one fully-featured app (Followgram, a self-styled Instagram directory) was built on top of this unofficial API. In January 2011, Instagram shut it down though, and released an official API in February.

In time its main features included the ability to search for pictures by hashtag or location, the ability to incorporate pictures into a website or a mobile app and to print photos. Within days, a host of photo sharing apps and mashups were released by third party developers. The more popular ones surviving to this day include Websta, Flipboard, Casetify, Gramfeed, Collecto (formerly Followgram and Social Print Studio.

Following the Facebook acquisition in 2012 (and given the need to monetize its hundreds of millions of users), Instagram has been able to branch out in new directions, notably a very successful ad API.

Like Twitter, Instagram has recently clamped down on its public API consumers, and has announced that it would now review apps before letting them use the API in production. This follows from the InstaAgent scandal in which an application was used to steal user credentials.