The Three Silent Failure States Every Streaming API Should Document

Posted in

Streaming data has a big upside over a normal request-response API. Instead of asking for the latest state over and over, you keep a connection open and receive changes as they happen.

The problem is that streams can give you a false sense of security when something goes wrong. A normal request returns, times out, or gives you an error. With a stream, the connection can still look healthy. Heartbeats might still arrive. Other data might still change. Meanwhile, the client is holding state that is incomplete or stale.

That is worth separating out, because “connected” does not tell you as much as people assume it does.

Failure Why it looks healthy What broke What to check
Transport The socket still looks open Data is no longer getting through Ping and Pong or heartbeat timeout
Continuity New messages are arriving Some updates were missed Sequence numbers, cursors, replay
Freshness The stream is busy One source behind it stopped refreshing Source observation time

1. The Connection Dies Quietly

This is the easiest one to understand: A connection dies without giving the client a clean indication that it has died. It could be the server, the user’s network, a proxy, a firewall, or a device changing networks. The application does not always get a WebSocket close event immediately, so the socket can still appear open even though nothing useful is coming through.

Take an odds feed. A price is sitting at 2.10 when the connection stops carrying updates. If the client does not notice for five minutes, it can keep displaying 2.10 as though that were a current price. The actual price may have moved several times.

That is why streaming APIs send heartbeats. With WebSockets, Ping and Pong frames can test whether the other endpoint responds. An API might also send its own heartbeat message. The documentation needs to say how often to expect it, how long the client should wait, and what to do when it stops.

If the API normally sends one every ten seconds and the client has seen nothing for 30, it has a good reason to stop treating the connection as healthy. Those numbers are only an example. The point is to give the client a limit it can implement.

Reconnecting fixes the transport problem. It does not tell you what happened while the connection was down, or whether the state you had before reconnecting is still correct.

2. The Stream Resumes, but the History Has a Hole

This is where the bigger problem starts. The connection comes back, data starts flowing again, and everything looks normal. But the client may have missed changes in between.

Say it received updates 410, 411, and 412 before disconnecting. The next update after reconnecting is 419. The socket works, but updates 413 through 418 are gone.

In a sports odds stream, one of those missing updates could have suspended a selection or removed a market. The client can be processing update 419 while the state underneath it is already wrong.

Sequence numbers, cursors, and resume tokens give the client a way to say, “This is the last update I know I had.” The server can then replay what happened after that point, if it has retained those changes. For Server-Sent Events, the browser can send a Last-Event-ID header when it reconnects. The API still has to define what that ID means and whether it can replay from there.

There is a cost to keeping every delta available. The longer a resume token remains valid, the more old updates the API has to hold. Make the window too short and clients keep falling back to a full snapshot. Make it too long and you are storing a heap of changes in case someone comes back. Mobile clients make this tradeoff obvious: People background the app for a few minutes and return all the time.

Replaying everything is not always the best answer either. We have run into cases where the server could send every missed update, but the receiver could not usefully process a large catch-up batch. If thousands of odds changes each trigger filtering, calculations, or a screen render, the app can freeze or fall behind again before it reaches the live edge.

At some point it is cheaper and cleaner to say, “This gap is too big. Give me a new snapshot.” There is no one number that works for every API. It depends on event size, update rate, and what the client does with each change. But the API should have a position on it: When do you replay, and when do you start over?

Watch the Gap Between Snapshot and Stream

A snapshot gives the client the full current state. The stream sends the changes after that. If the client cannot recover a missing set of deltas, a new snapshot gets it back to a state it can trust.

There is one annoying edge case. Say the snapshot represents version 500. While that request is being returned, versions 501 through 505 happen. The stream opens at 506. The client has now missed five updates before it really started.

That is why the snapshot and stream need a shared version or cursor. One safe flow is to open the stream first, buffer its updates, fetch the snapshot, then apply only the buffered updates newer than the snapshot. Another is to open the stream from the snapshot’s cursor, if the API supports it. If the client cannot close the gap, it needs to start again.

It sounds like a small implementation detail. It is exactly the sort of thing that creates hard-to-find state issues later.

3. The Stream Is Busy, but One Source Has Stopped

This is the one I think gets missed most in larger data systems. The stream itself can stay extremely busy even when part of the data behind it has stopped updating.

Say an odds API carries 160 bookmakers and one bookmaker’s feed stops refreshing. The other 159 keep working. The customer still gets thousands of updates. Heartbeats arrive. The WebSocket is connected. Continuity can look fine. But that one bookmaker’s prices might be 20 minutes old.

Checking when the client received its last message tells you nothing about that specific source. The message could have come from any of the other 159 bookmakers. What matters is when the source you care about was last successfully observed.

There is a trap in using the last time a value changed. An unchanged price is not automatically stale. A price could sit at 2.10 for 30 minutes while the collector checks it every five seconds. That is still fresh data. Compare that with a price that last changed 30 minutes ago and was last observed 30 minutes ago. The same number is on screen, but it may just be an old cached value.

That is why last_observed_at is a better freshness signal than last_changed_at. The API also needs to say what it applies to: a source, event, market, or individual price. How often should it be observed? When is it stale? A client should not have to infer one source’s health from unrelated traffic on the same socket.

“Connected” Is Not a Data-Health Check

A lot of streaming implementations reduce health to connected or disconnected. That misses the three questions the client actually needs answered:

  • Is the connection working?
  • Did I miss anything?
  • Is the data itself still fresh?

A client can answer yes to the first and still be wrong on the other two. That is where people get caught.

Document the heartbeat timeout, the scope and lifetime of a resume cursor, when replay gives way to a new snapshot, how the snapshot meets the stream, and when each source was last observed. Tell the client what it should display or do when it cannot prove the answer.

If the socket closes, you know it closed. The harder failures are the ones where the stream keeps moving, the UI still says live, and the client has already lost the correct state.

AI Summary

This article explains three silent failure states in streaming APIs — transport, continuity, and freshness — and what API providers should document so clients can detect stale or incomplete state while a connection still looks healthy.

  • A connection can die without a clean close event, so streaming APIs should document heartbeat intervals, how long clients should wait, and what clients should do when heartbeats stop.
  • Reconnecting restores transport but not continuity. Sequence numbers, cursors, and resume tokens let clients detect gaps and request replay of missed updates.
  • Replay windows trade storage cost against snapshot fallbacks, and large catch-up batches can overwhelm clients, so APIs should define when to replay and when to start over with a new snapshot.
  • Snapshots and streams need a shared version or cursor to avoid missing updates that occur while a snapshot is being returned.
  • A busy stream can hide a single stale source, and a per-source last_observed_at timestamp is a better freshness signal than last_changed_at.

Intended for API providers, API architects, and developers building or consuming real-time streaming APIs over WebSockets or Server-Sent Events.