The rise of remote work, necessitated by the pandemic and now under attack, has fueled an obsession among managers everywhere: productivity. In August 2023, McKinsey published an article about how to measure developer productivity. Generally speaking, developers did… not like it.

Critics argue that attempting to boil down the output of a software developer to a single productivity metric, or collection of metrics, is doomed to fail. They make the case that prioritizing, say, deployment frequency will only lead to buggier code and broken products.

And, to be fair to McKinsey, they do acknowledge that point in their article: “In software development, the link between inputs and outputs is considerably less clear…[It] is also highly collaborative, complex, and creative work and requires different metrics for different levels.”

Despite that, they ultimately reach the conclusion that even though “building a developer productivity initiative can seem daunting… there is no time like the present to begin to lay the groundwork.” In other words, the productivity movement is not going away any time soon.

Which begs the question: Can you really measure API developer productivity? And perhaps an even better question: Should we? Below, we’ll endeavor to answer both of these questions.

DORA the (SPACE) Explorer

Before we can get into measuring productivity in API development, let’s look at some of the metrics suggested for measuring overall software development productivity. These are often called the DORA metrics because they were proposed by the DevOps Research and Assessment (DORA) team at Google:

• Deployment frequency: How often changes are shipped.
• Lead time for changes: The time it takes for commits to reach production.
• Change failure rate: The percentage of releases that fail, leading to downtime or rollbacks.
• Time to restore service: The average time it takes to respond to issues.

Although these seem like positive indicators of agility and productivity in isolation, it’s worth noting that there’s some inherent conflict between them. Amping up deployment frequency, for example, is likely to have a negative impact on change failure rate if things are rushed.

In 2021, GitHub researchers suggested another acronym, SPACE, designed not to replace DORA metrics but to complement them. Here’s what those letters stand for:

• Satisfaction and well-being
• Performance
• Activity
• Communication and collaboration
• Efficiency and flow

Where DORA seeks to measure the productivity of a development team, SPACE ostensibly zeroes in on measuring individual performance. We’ll get into how useful (or not, as the case may be) trying to do that is below, given that developers rarely work in isolation.

SPACE outlines a broad set of attributes that are not easy to measure. Methods for gauging employee satisfaction, for example, are often called into question because of how easily they can be manipulated. Their output is more likely to be qualitative than quantitative, so it can (consciously or unconsciously) be skewed by employers, employees, or both.

However they aim to do it, anyone measuring developer productivity faces an uphill battle. There’s no silver bullet for measuring developer productivity… which is exemplified by the vagueness of the SPACE framework and friction between DORA metrics.

So, does the situation get any easier in the context of APIs?

Productivity and the API Development Lifecycle

In our article on API metrics to track when productizing APIs, we include the following for building a picture of an API’s performance:

• Time To First Hello World (TTFHW)
• Time To First Value (TTFV)
• Number of active monthly users
• Revenue growth
• Churn rate
• Requests per minute (total/per client)
• Error rates
• Uptime

Ease of use and reliability are things that we come back to time and time again when talking about creating an effective API. Of the above, the first four or five (adoption and retention metrics) speak to ease of use, while the later entries (mostly operational metrics) lean closer to reliability.

Reliability has been a key factor in changing the perception of APIs from fun side projects to legitimate business tools. Coincidentally, its importance was also highlighted by DORA when they later added reliability to their list of productivity metrics.

Early in 2023, we wrote extensively about the concept of API lifecycle management. We covered things like the utility of the OpenAPI Specification for planning APIs and using apps like ReadyAPI or Postman for API testing. Could these also be used to track productivity?

Theoretically, creating KPIs around these points — % adherence to OpenAPI Specs, number of Postman collections created, or successful linting of APIs — could be used to track API developer productivity… or something close to it. And following uptime and churn rate could give a broader picture as well.

Examples of Productivity Measurement In Practice

It’s tough finding examples of API providers measuring developer productivity. But we can look at some other references for more context. For instance, 7pace suggests a range of sample metrics for measuring developer productivity, including pull request size, code coverage, time spent fixing bugs/errors, and time-to-open. They’re not all relevant to API developers. Still, we approve their suggestion of using “metrics with a strong correlation with business outcomes (e.g. revenue)…while being resistant to people gaming the system.”

Currently, it seems that many companies are using simpler, more traditional metrics than these. A Vox article on productivity, for example, suggests that Hubspot records the “amount of time that software engineers spend revising code.” (To Hubspot’s credit, though, they do seem to be investing resources in building a connected workforce that embraces WFH and hybrid working, rather than using productivity as an excuse to bring workers back into the office.)

In a blog post on employee productivity, tech brand Doist suggests “standardized yet flexible” goal–setting for benchmarking productivity, like “releasing minimum-viable products on time,” and helping team members to find work that aligns with their strengths.

But, Doist also acknowledge what many of us are thinking right now:
“The old ‘productivity = output divided by input’ model fails us on many fronts.”

The Future of (API) Developer Productivity

We should point out that, although our suggestions above measuring API performance are a little more involved, they don’t so much measure developer productivity as they measure developer output. It’s a minor distinction based on the following assumption:

Any team putting out high-performance APIs is, by definition, a productive one.

Much of the pearl-clutching around productivity right now — often centered around remote working and the use of AI in writing code — may come to be seen as overkill in the future. Not least because obsessing over productivity can be harmful when policed too harshly.

As a former boss told me not long after the company introduced flexi-time, remote working, and an unlimited vacation policy: “I don’t mind how your work gets done, as long as it gets done, and done to the best of your ability.” Productivity was based on outcomes, not processes.

We can get around productivity being difficult to measure by shifting focus to the quality of the end result. This can be facilitated by an API-as-a-Product mindset, where measurables include:

• Number of successful API integrations
• Durability of the platform
• Overall satisfaction of API consumers*
• Revenue that an API platform brings in
• The degree of breaking changes new development produces

Of course, developer satisfaction can also be tricky to measure! For example, it’s tough to quantify the success of developer documentation, but we know that great API documentation can be a game-changer for API adoption.

Let’s be clear: productivity is important. We understand that managers are worried their development team is sitting around doing nothing, especially because many C-level execs don’t have coding experience. But creating boxes to tick is probably not the right way to remedy that.

When developers are given rewarding and challenging work — and, let’s face it, API development can be both rewarding and challenging — they’ll engage with that work. And it will quickly become apparent if a developer isn’t engaging for hours, or days, on end. Address that then, not before.