No developer likes an API that takes too long to respond. Worse, poor service uptime makes it challenging to rely on APIs for stable integrations. Low quality can stunt the developer experience and lead to unhappy end consumers. As such, retaining high performance is key to creating successful API products.

Yet, after years of steady improvements, overall cloud API performance seems to have reached a ceiling. This partly stems from variances across cloud hosting service providers and their regions. However, it could also be due to higher processing demands due to new working conditions and new technology trends, like generative AI.

To explore the state of cloud API performances, APIContext recently released its 2024 Cloud Service Provider API Report. Now in its sixth year, the research analyzed 650 million API calls to 1,000 APIs across 100 geographically diverse data centers. With such a wealth of data, the report is a window into global API usage and performance trends throughout the last year. Let’s dive into some of the key findings.

Service Availability Degradation

There has been a noticeable statistical degradation in cloud latency. In 2023, only 7% of API services globally achieved a 99.99% service availability score, a significant drop from 18% in 2022. Yet again, PagerDuty was the API service with the highest observable availability, with just 30 minutes of measurable downtime across its APIs.

Part of the increase in latency is perhaps due to the at-home remote working conditions brought on by the pandemic, suggests the report, which, for many organizations, are still in full force. API reliance has also shot through the roof, as organizations are utilizing more APIs than ever.

“While we see an overall high-quality standard as the baseline, the increased traffic and continual rollout of new physical infrastructure has put high demands on both cloud hardware and teams,” says Jamie Beckland, Chief Product Officer, APIContext. “We also see that cloud customers — from individual developers to large enterprises — do not often pursue credits and refunds for SLA violations.”

New Technology Brings New Cloud Demands

In 2023, global new data center capacity increased by 19%, but demand for cloud services increased by 20%. This condition is putting pressure on clouds to meet SLAs, typically set at 99.5%, says Beckland. And, in 2023, we witnessed the outcomes of global computing demands — all clouds were slower in terms of total time to complete requests.

Another factor to consider is the heavy reliance upon cloud computing for new artificial intelligence and machine learning development. “Cloud AI models grew much faster than overall cloud services,” says Beckland. “AI is taking more of the overall cloud capacity, and it’s also taking the fastest and most advanced hardware.”

With the increasing interest in AI, consuming cloud resources for AI is anticipated to outpace the rest of the cloud entirely. This will likely worsen over time, predicts Beckland, but it won’t necessarily affect DNS or transit times of APIs. “The biggest impact is felt by application processing time, which is the place where API product owners have the most ability to impact their customers’ experience.”

Geographic Differences in Data Centers

Performances continue to vary across cloud service providers. But if you’re considering a cloud hosting provider solely based on high performance, Amazon Web Services (AWS) is still a very safe bet. In 2023, AWS led the pack in terms of connection time, with four out of the top five performant locations being AWS data centers. Azure, on the other hand, was consistently 75 milliseconds slower globally than AWS.

But the report didn’t just expose differences across cloud providers — it also measured changes across geographies. Interestingly, the AWS East Virginia location was narrowly usurped by Asia-Pacific Northeast in 2023, which had an average time to connect of 2.50 ms and 2.28 ms, respectively. This indicates technology expectations in the region are catching up and surpassing those of North America.

Overall API Quality Plateaus

Overall, API services had good observable performance in 2023, as measured by APIContext’s homebrewed Cloud API Service Consistency (CASC) score. The report found that 68% of providers had a CASC score between 8.00 and 8.99, and 32% had a CASC score between 9.00 and 9.99.

That said, every extra millisecond of latency counts. The impact of slow DNS times, for instance, can quickly accumulate, especially within interconnected API flows that leverage dozens of API calls, says Beckland. “DNS lookup time is fundamental to every API call on the internet because it routes every request to the appropriate resource.”

Naturally, API performance is partially at the hands of the cloud provider’s performance. In 2023, we saw DNS resolution slow for Google and IBM. As a result, APIs like those from Box, DocuSign, and Capitol One were found to have significantly higher DNS lookup times. In comparison, quick DNS seems to be a competitive advantage for AWS. “It’s clear that AWS has focused on DNS connect performance as a “pure play” differentiator from other cloud providers,” says Beckland.

Takeaways For API Providers

While cloud-based API performance quality is generally high, there are some indicators that overall performance has reached a ceiling. To move the dial forward on performance, there are steps API providers can take to optimize their designs, such as caching, optimizing queries and payloads, pagination, and other techniques. However, taking performance to the next level will also depend on assessing individual cloud providers and their commitments to areas like high availability and optimized DNS.

For Beckland, this strategy will necessitate a hybrid, multi-cloud approach. “API providers should update their infrastructure to operate more smoothly in hybrid cloud environments,” he says. “Hybrid environments allow APIs to be optimally routed based on the location and character of specific requests and responses, which allows API owners to use the best components of each cloud provider. As a bonus, this approach often saves operational costs.”