Most people are familiar with the famous quote from Marc Andreessen that “software is eating the world.” Now, APIs are eating software.

As APIs become ubiquitous, developers are demanding real-time event data from their API providers. And a common solution for real-time updates from APIs is API polling.

However, polling comes with some downsides. As Zapier, the popular low-code/no-code integration platform describes:

“Polling is the process of repeatedly hitting the same endpoint looking for new data. We don’t like doing this (it’s wasteful), vendors don’t like us doing it (again, it’s wasteful) and users dislike it (they have to wait a maximum interval to trigger on new data). However, it is the one method that is ubiquitous, so we support it.”

Enter webhooks.

Webhooks allow API users to receive a notification when specific events occur. This eliminates the need to send constant requests to API providers looking for updates.

Many of the top API providers (think Stripe, Sendgrid, etc.) have extensive and well-documented webhook features that make receiving real-time updates a breeze for their users.

Why Webhooks?

Here are four reasons why your API should offer a webhooks feature and stop relying on API polling.

Resource Efficiency

Generally, a webhook system will be much less resource-intensive than a polling system. This is true for both the API provider and the consumer.

For example, if 1,000 users poll the API every 5 seconds, your API will have to be able to handle up to 1,000 requests per second, depending on the timing of the requests.

Zapier estimates that only 1.5% of polling requests find an update. That implies that with a webhook solution, you would only need to send up to 15 responses per second while your users don’t have to send any requests at all. That’s less than 1% of the resources needed to sustain the polling solution (1000 requests + 1000 responses).

Customer Experience

Webhooks also offer a better customer experience. Forcing your users to constantly poll your API for updates means they need to do the heavy lifting of maintaining and comparing states to determine when and if changes have occurred.

Think about the common situation of a family on a road trip. Polling is equivalent to the kids constantly asking, “are we there yet?” and the parents replying “no” repeatedly. Wouldn’t you prefer the webhook scenario where the kids are quietly reading a book or watching a movie while the parents get to drive in peace?

Real-Time Updates

Our example scenario only accounts for one request every five seconds. But what if customers want updates in real-time? They would need to send requests at least every second, if not more. This increases the load on your system and increases operational complexity.

If the API provider has rate limits, they might block you from sending requests due to exceeding the specified rate limit. If the customer has implemented polling at frequencies that do not exceed the rate limit, they end up polling so infrequently that their updates are stale. This can decrease customer satisfaction and lead to churn as your API could no longer be a valid solution.

Webhooks avoid this issue by sending updates right as events occur.

Developers Expect Webhooks

Major API providers like Stripe and Plaid have very well-documented webhook features, and they are becoming commonplace with the most popular APIs. That means developers are used to working with the webhook pattern as they are fairly simple for customers to ingest. The more other APIs start offering webhooks, the more developers will expect APIs to offer webhooks.

Webhooks also improve the developer experience. In fact, a poll conducted by Wufoo (now part of SurveyMonkey) found that 82% of developers preferred working with webhooks over API polling.

Easy Integration

Zapier, IFTTT, and Make (formerly Integromat) are common ways for users to integrate their apps. Both work using API polling and webhooks (and we already know how we feel about polling).

Integrating with these workflow automation platforms has many benefits:

• Integrate with 3000+ apps currently available in the marketplace
• Continued integration with any new apps added
• Increased visibility of your API to 3.5M+ users
• Expand users beyond developers by leveraging no-code integration

So Why Aren’t More APIs Offering Webhooks?

There are some cases where polling would be a preferable solution to webhooks. When updates are more frequent than the polling interval, it’s more efficient to get the updates in batches via polling. Polling is a fine solution if you also don’t need the updates in real-time.

The main thing blocking most API providers from offering webhooks is that they’re tough to implement at scale. Unreliable user endpoints mean you need automatic retries to ensure deliverability. Monitoring deliverability and notifying customers when their endpoints are broken is critical. There are many potential security vulnerabilities like server-side request forgery (SSRF) and replay attacks that you need to defend against. You need a way for your users to authenticate webhook events to ensure they are coming from your API. And, it helps to have a UI for your users to test, debug, and monitor their implementations.

You can see all the potential components and services needed to build a secure, reliable, and scalable webhook system in this architecture diagram:

Even Lob, a well-funded Silicon Valley startup with a strong engineering culture, recently revamped its system using a webhook service. They successfully simplified the design to make their engineers’ lives easier, increase scalability, add features to their dashboard, and make their cost structure more predictable.

Summary

Webhooks are becoming a must-have feature for APIs. They’re very efficient for providing real-time updates, offering an excellent developer experience, and enabling many potential integrations through low-code/no-code platforms like Zapier and Make.

Although they can be difficult to implement at scale, there are open source and hosted solutions that make it simple to offer a great webhook experience for your users.