Webhooks, WebSockets, pub/sub or polling: choosing by delivery guarantees and failure modes

Sooner or later every system needs to react to something that happened somewhere else: a payment succeeded, a file finished processing, a price changed, a teammate edited a document. There are four common ways to find out: ask repeatedly (polling), be called back (webhooks), keep a connection open (WebSockets and similar), or go through a broker (pub/sub). Each is the right answer somewhere, and each fails in its own characteristic way.

This guide compares them by the properties that actually decide the choice, then walks through the combinations we use in practice.

The four options in one table

Polling

The consumer asks "anything new?" on a schedule: GET /orders?updated_since=....

Strengths: the simplest to build and reason about; works through any firewall; the consumer controls the pace; nothing is lost if the source keeps state, because the consumer can always ask again from its last checkpoint.

Failure modes:

• Latency versus load: frequent polling wastes requests when nothing changes; infrequent polling adds delay.
• Missed updates if the query uses timestamps carelessly: two updates in the same second, clock skew, or rows committed late with an earlier timestamp. Use a monotonically increasing cursor (sequence number or change id) instead of updated_at where possible.
• Thundering herds when many clients poll on the same schedule. Add jitter.

Use when: updates are infrequent or latency of a minute is fine, the consumer cannot expose an endpoint, or as a reconciliation safety net behind any other mechanism.

Webhooks

The producer sends an HTTP request to a URL the consumer registered, when the event happens.

Strengths: low latency without wasted requests; universal (any HTTP stack can receive); the standard for integrations between companies (payments, source control, messaging).

Failure modes:

• The consumer is down: the producer must retry with backoff. Many providers retry for hours or days; some do not retry at all. Know which.
• Duplicates and reordering: retries mean the same event may arrive twice and out of order. Consumers must be idempotent, keyed by event id, and handle older events arriving after newer ones.
• Spoofing: anyone who knows the URL can send requests. Verify signatures (usually HMAC over the raw body with a shared secret) and timestamps.
• Slow handlers: providers time out after a few seconds. Acknowledge quickly, then process asynchronously.

@app.post("/webhooks/payments")
async def payment_webhook(request: Request):
    raw = await request.body()
    verify_signature(raw, request.headers["Webhook-Signature"], SECRET)   # reject if invalid
    event = json.loads(raw)
    if await seen_before(event["id"]):
        return Response(status_code=200)                                  # duplicate, already handled
    await queue.enqueue("payment-events", event)                          # process later
    return Response(status_code=200)

Use when: integrating with external services, server-to-server notifications where the consumer can host an endpoint, and moderate event volume.

WebSockets and server-sent events

The client opens a long-lived connection, and the server pushes messages over it. WebSockets are bidirectional; server-sent events (SSE) are one-way over plain HTTP and simpler when the client only listens.

Strengths: very low latency; works for browsers and mobile apps, which cannot receive webhooks; ideal for live interfaces such as chat, dashboards, collaborative editing and progress bars.

Failure modes:

• Disconnects are normal: mobile networks, laptops sleeping, deploys restarting servers. Anything sent while a client is disconnected is lost unless the protocol supports resuming, for example with event ids and a "since" parameter on reconnect (SSE has Last-Event-ID built in).
• Reconnect storms: when a server restarts, every client reconnects at once. Use jittered backoff on the client.
• Scaling state: each server holds its own connections, so a message for a user must reach the server that holds that user's connection, usually through a pub/sub layer behind the servers.
• Proxies and load balancers with idle timeouts close quiet connections; send heartbeats.

Use when: users need updates in seconds or less in a browser or app, and you can accept that the connection is a best-effort channel backed by a source of truth.

Pub/sub through a broker

Producers publish events to a broker (Kafka, a cloud pub/sub service, RabbitMQ, Redis streams, a cloud event bus), and consumers subscribe.

Strengths: decouples producers from consumers; native fan-out; persistence and replay in many brokers; consumers process at their own pace with backpressure.

Failure modes:

• Consumer lag: slow consumers fall behind; you need lag monitoring.
• At-least-once delivery: duplicates happen; consumers must be idempotent.
• Ordering is usually guaranteed only within a partition or key.
• Operational cost of running or paying for the broker, and of schema management as events evolve.

Use when: many internal services react to the same events, volume is high, or you need replay and durable buffering between systems you control.

Common combinations

Real systems usually combine mechanisms, each covering another's weakness.

The pattern behind all of them: one mechanism for speed, another for correctness. Push mechanisms are fast but can lose or duplicate messages; a pollable source of truth lets any consumer recover.

Decision questions

1. Can the consumer expose a public endpoint? If not, webhooks are out.
2. Is the consumer a browser or app? Then WebSockets or SSE for live updates, polling otherwise.
3. How fast must updates arrive? Seconds or less rules out slow polling.
4. How many consumers per event? Many suggests a broker.
5. What happens if an update is missed? If that is unacceptable, add reconciliation regardless of the primary mechanism.
6. Who operates what? A broker is a system to run; a webhook endpoint is an attack surface to protect.

Key takeaways

• Polling is simple and robust but trades latency for load; use a cursor, not timestamps.
• Webhooks are the integration standard; verify signatures, be idempotent, acknowledge fast.
• WebSockets and SSE serve live interfaces; treat them as best effort and support resume.
• Pub/sub decouples internal services with fan-out and replay, at operational cost.
• Combine a fast push mechanism with a pollable source of truth for correctness.

References

• MDN, WebSockets API: https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
• MDN, Using server-sent events: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
• Standard Webhooks specification: https://www.standardwebhooks.com/
• RFC 6455, The WebSocket Protocol: https://www.rfc-editor.org/rfc/rfc6455