What are choreographed events and synchronous calls, and how do they differ?

In a synchronous call, one service sends a request and waits for the response before it can continue. The caller gets an answer it can act on immediately. In choreographed events, a service publishes a fact such as OrderPlaced to a broker and moves on. Other services subscribe and react independently, and no central coordinator directs the sequence.

The core difference is temporal coupling: whether both sides must be running at the same moment for the interaction to succeed. With request-response, if the inventory service is down or slow, checkout is down or slow too. With events, the broker holds OrderPlaced until inventory recovers and consumes it. Checkout never notices the outage.

Synchronous chain               Choreographed events

Checkout                        Checkout --publish--> [ Broker ]
   |  waits                                             |   |   |
   v                                                    v   v   v
Inventory                                   Inventory Payment Email
   |  waits                                 (each subscribes and
   v                                         consumes on its own
Payment                                      schedule)
   |  waits
   v
Shipping

Temporal coupling multiplies availability along a chain. Illustrative estimate: assume four services in a synchronous chain, each independently available 99.9% of the time. The chain succeeds only when all four are up, so its availability is 0.999⁴ ≈ 99.6%. Over a 30-day month, that is roughly 2.9 hours of expected failure instead of about 43 minutes for one service. Adding a service to a synchronous chain lowers the availability ceiling of every caller upstream of it.

 SYNCHRONOUS CHAIN: caller waits on every hop (temporal coupling)
 +----------+     +-----------+     +---------+     +----------+
 | Checkout |---->| Inventory |---->| Payment |---->| Shipping |
 |  waits   |<----|   waits   |<----|  waits  |<----|          |
 +----------+     +-----------+     +---------+     +----------+
 Illustrative estimate: 0.999^4 = ~99.6% available end to end
 (~2.9 h expected failure per 30-day month vs ~43 min for one service)

 CHOREOGRAPHED EVENTS: publisher does not wait for subscribers
 +----------+          +-------------------------------------+
 | Checkout |-publish->|  Broker: OrderPlaced is durably     |
 +----------+          |  stored until each subscriber reads |
   moves on            +-------------------------------------+
                              |             |           |
                              v             v           v
                        +-----------+  +---------+  +-------+
                        | Inventory |  | Payment |  | Email |
                        +-----------+  +---------+  +-------+
                        each consumes on its own schedule

In a synchronous chain every caller waits on the hops below it, so availability compounds (illustrative estimate: four services at an assumed 99.9% each give about 99.6%), while with choreographed events the broker holds OrderPlaced so each subscriber can consume it independently.

The per-interaction decision test: who should use which style and when

That ceiling is a cost worth paying only when the caller cannot proceed without an answer. The test for any interaction comes down to one question: does the caller need the result to continue its own work, or is it announcing a fact that has already happened? If it needs the result, use a synchronous call. If it is announcing a fact, publish an event and let interested services react on their own schedule.

Interaction Caller needs result to continue? Style Why
Price lookup during checkout Yes Synchronous request-response The total shown to the customer depends on it
Payment authorization Yes Synchronous request-response The order cannot be confirmed until the card is approved or declined
Order placed No Event The order already exists; inventory, shipping and analytics react independently
Email sent No Event It is a record of something done; nothing upstream waits on it

Notice that one checkout flow contains both styles. Calling the whole system "event driven" or "synchronous" hides this, and it pushes teams to force a payment authorization through a queue or to make order fulfilment wait on a chain of blocking calls. Temporal coupling is a property of each edge between services, so the decision belongs to each edge.

The team that owns the producing service should make the call, because it knows whether the interaction is a question or a statement. Consumers should review it, since they inherit either the latency or the eventual consistency.

How to implement synchronous calls responsibly

A synchronous call only works if the caller has decided in advance how long it will wait and what it will do when the answer doesn't come.

Set a timeout on every call. Base it on the callee's observed tail latency and on the budget left in the user request, not on a library default. Without a timeout, one slow dependency holds threads and connections until the caller fails too. Pass the remaining deadline downstream so inner services don't keep working on requests the edge has already abandoned.

Retry rarely and with limits. Retry only errors that are likely to be transient, such as connection resets, 503s and timeouts, and set a small cap on attempts. Use exponential backoff so a struggling service gets time to recover. Add jitter so many clients don't retry at the same moment. If several layers each retry, the number of attempts multiplies, so pick one layer to own retries.

Add a circuit breaker. After repeated failures, a breaker fails fast for a cooldown period instead of sending more traffic to a dependency that is already down. This protects the callee and returns errors to users sooner.

Make retried writes safe. A timeout doesn't tell you whether the write succeeded. Send an idempotency key that the client generates once per logical operation and reuses on every retry. The server stores the result under that key and returns it again instead of applying the write twice.

Keep user-facing chains shallow. Each extra hop adds to latency and to the chance of failure. If a request fans out three levels deep, consider whether some of those calls should be events.

// Pseudocode: one logical write, safe to retry
function createPayment(order, requestDeadline):
    // Generate the key ONCE per logical operation and reuse it on every attempt,
    // so the server can return the stored result instead of charging twice
    idempotencyKey = newUUID()

    maxAttempts = 3            // small cap; this is the only layer that retries
    baseDelayMs = 100
    perCallTimeoutMs = 800     // from callee's observed tail latency, not a default

    for attempt in 1..maxAttempts:
        remainingMs = requestDeadline - now()
        if remainingMs <= 0:
            return error("deadline exceeded")   // the user has already been abandoned

        timeoutMs = min(perCallTimeoutMs, remainingMs)
        try:
            response = http.post("",
                body    = order,
                headers = {
                    "Idempotency-Key": idempotencyKey,
                    // pass the remaining budget so downstream can stop early
                    "X-Request-Deadline-Ms": remainingMs
                },
                timeout = timeoutMs)

            if response.status in [200, 201]:
                return response.body
            if response.status != 503:
                return error(response)          // 4xx etc. will not fix themselves
        catch ConnectionReset, Timeout:
            pass                                // transient: may retry

        if attempt < maxAttempts:
            // exponential backoff with full jitter to spread retries out
            sleep(random(0, baseDelayMs * 2^(attempt - 1)))

    return error("payments unavailable after retries")

Pseudocode for a client write that bounds each call by a timeout and the remaining request deadline, retries only transient failures a few times with jittered exponential backoff, and reuses one idempotency key across attempts so a retried write is not applied twice.

How to implement events reliably: outbox, idempotent consumers and event design

The core hazard is the dual write. A service that updates its database and then publishes to a broker can fail between the two steps. The result is a committed order with no event, or an event for an order that rolled back. No retry logic fixes this, because the two systems share no transaction.

Transactional outbox

Write the event into an outbox table in the same transaction as the business change. A minimal schema has an id, aggregate_id, event_type, payload, created_at and a nullable published_at. A separate relay polls for unpublished rows in order, sends them to the broker, and marks them published. Because the relay can crash after sending but before marking, delivery is at-least-once, so consumers must tolerate duplicates.

Idempotent consumers

Give every event a stable ID and have consumers record processed IDs in a dedupe table, in the same transaction as their own state change. Where possible, apply effects as upserts keyed on the entity, so replaying an event converges to the same row.

Ordering is usually guaranteed only within a partition or key. Partition by aggregate ID if per-entity order matters, and expect no global order.

Event design

Thin events ("order 42 changed") force consumers to call back, which reintroduces temporal coupling. Fat, state-carrying events avoid the callback but make the payload a public contract. Evolve that contract additively: add optional fields and never rename or repurpose existing ones. Avoid publishing internal fields. Once a consumer depends on one, you can no longer change it without coordinating with that consumer.

-- PostgreSQL. Outbox table lives in the producer's own database.
CREATE TABLE outbox (
  id           uuid PRIMARY KEY,          -- stable event ID, reused by consumers for dedupe
  aggregate_id text NOT NULL,             -- used as the broker partition key for per-entity order
  event_type   text NOT NULL,
  payload      jsonb NOT NULL,
  created_at   timestamptz NOT NULL DEFAULT now(),
  published_at timestamptz                -- NULL until the relay has sent it
);
CREATE INDEX outbox_unpublished ON outbox (created_at) WHERE published_at IS NULL;

-- Producer: business change and event commit or roll back together, so there is no dual write.
BEGIN;
UPDATE orders SET status = 'PAID' WHERE id = 42;
INSERT INTO outbox (id, aggregate_id, event_type, payload)
VALUES ('3f2b8c1e-7a4d-4e1b-9c2f-5d6e7a8b9c0d', '42', 'OrderPaid',
        '{"orderId": 42, "status": "PAID"}');
COMMIT;

-- Relay loop (single instance): read pending rows in order, publish keyed by aggregate_id,
-- then mark them. A crash between publish and UPDATE resends them: at-least-once delivery.
SELECT id, aggregate_id, event_type, payload
FROM outbox WHERE published_at IS NULL ORDER BY created_at LIMIT 100;
UPDATE outbox SET published_at = now() WHERE id = ANY($1);  -- $1 = IDs the broker acknowledged

-- Consumer (its own database): dedupe record and state change in one transaction.
CREATE TABLE processed_events (event_id uuid PRIMARY KEY);
BEGIN;
INSERT INTO processed_events (event_id)
VALUES ('3f2b8c1e-7a4d-4e1b-9c2f-5d6e7a8b9c0d') ON CONFLICT DO NOTHING;
-- If that inserted 0 rows, this is a duplicate: ROLLBACK and acknowledge the message.
INSERT INTO order_view (order_id, status, updated_at)
VALUES (42, 'PAID', now())
ON CONFLICT (order_id) DO UPDATE                -- upsert: replays converge to the same row
  SET status = EXCLUDED.status, updated_at = EXCLUDED.updated_at;
COMMIT;

A PostgreSQL sketch of a transactional outbox: the producer writes the event in the same transaction as the business change, a relay publishes pending rows, and an idempotent consumer combines a dedupe table with an upsert.

Designing for eventual consistency and observability

When a command triggers an asynchronous flow, the API should report what actually happened: the request was accepted, and the work has not finished. Return 202 Accepted with a Location header that points to a status resource, such as /orders/8f3a/status, and a body like {"orderId": "8f3a", "state": "processing"}. Clients poll that resource or subscribe to push updates through webhooks, WebSockets or server-sent events. Model the states explicitly (processing, confirmed, rejected) so the UI can show progress and failures. Returning 200 here would claim the work is done when it isn't.

Read-your-own-writes is a product decision. Ask whether a user who just placed an order must see it in their order list immediately. If they must, the UI can render the result optimistically from the command it sent, or the service that accepted the write can serve that user's reads directly until the projection catches up. If a few seconds of lag is acceptable, document it so nobody files it as a bug.

Observability breaks easily at the broker. Trace context in HTTP headers is lost unless producers copy it into message headers. A published event should carry traceparent (the W3C Trace Context header), a correlation-id tied to the original user request, and a causation-id that names the message that triggered it. Consumers continue the trace and pass all three on to any events they publish. With these headers in place, you can reconstruct the full flow from the original HTTP request through every downstream reaction. Without them, each service's logs show only its own piece, with nothing to link them together.

# 1) Client sends a command. The trace starts at the edge.
POST /orders HTTP/1.1
Host: api.example.com
Content-Type: application/json
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
X-Correlation-Id: req-7d2c91e0

{"items": [{"sku": "BK-1042", "qty": 1}]}

# 2) The API accepted the work but has not finished it, so it returns 202
#    and points the client at a status resource it can poll.
HTTP/1.1 202 Accepted
Location: /orders/8f3a/status
Content-Type: application/json

{"orderId": "8f3a", "state": "processing"}

# 3) The OrderPlaced event published to the broker. Trace context is copied
#    into message headers, because HTTP headers do not cross the broker.
#    Same trace-id, new parent span-id for the publish operation.
headers:
  traceparent:    00-4bf92f3577b34da6a3ce929d0e0e4736-b7ad6b7169203331-01
  correlation-id: req-7d2c91e0        # ties every message back to the user request
  causation-id:   req-7d2c91e0        # what triggered this message (here, the HTTP request)
  message-id:     msg-0001-orderplaced
payload:
  {"type": "OrderPlaced", "orderId": "8f3a"}

# 4) A payment consumer reacts and publishes its own event. It keeps the
#    trace-id and correlation-id, and sets causation-id to the event it consumed.
headers:
  traceparent:    00-4bf92f3577b34da6a3ce929d0e0e4736-e457b5a2e4d86bd1-01
  correlation-id: req-7d2c91e0
  causation-id:   msg-0001-orderplaced
  message-id:     msg-0002-paymentauthorized
payload:
  {"type": "PaymentAuthorized", "orderId": "8f3a"}

An illustrative async flow: a 202 Accepted response with a status Location, then two events whose headers carry W3C traceparent, correlation-id and causation-id so the chain from HTTP request to downstream reactions can be reconstructed.

When to avoid choreography, and the alternatives: hybrids and orchestration

Even with full tracing, choreography has failure modes that tooling only partly fixes.

The end-to-end workflow exists only as the sum of subscriptions. No file or service describes "place order," so changing it means reading every consumer. As subscribers multiply, one new listener on OrderPlaced can trigger emails, charges or inventory moves that the publishing team never anticipated. Failures also surface late. A consumer three hops away rejects the event minutes after the user saw "success." A single malformed message can block a partition if the consumer retries it forever, stalling every event queued behind it, so route it to a dead-letter queue after bounded retries. Compensations in a choreographed saga are fragile because each service must infer from events alone whether to undo its step. If a step simply never happens, nothing notices unless you add timeouts and a reconciliation job that finds stuck instances.

Two alternatives reduce this. First, validate synchronously at the edge, such as stock, credit and input, so the caller gets a real answer, then publish events only for side effects that may lag. Second, once a flow passes roughly four or five steps or needs ordered compensation, move to orchestration. A single coordinator owns the state machine, the timeouts and the rollback.

Concern Request-response Choreography Orchestration
Workflow visibility In caller code Scattered across consumers Central definition
Temporal coupling High Low Low to medium
Failure surfaces Immediately Late, downstream At coordinator
Compensation Caller handles Each service infers Coordinator drives
Best fit Queries, validation Independent side effects Multi-step sagas

Impact on development cost and engineering time

Neither style is cheaper overall. Each one moves engineering cost to a different part of the system, so ask which cost your team is better equipped to carry for this interaction.

Cost area Synchronous calls Choreographed events
Infrastructure ownership Low: load balancers, retries, timeouts High: broker capacity planning, retention, consumer lag alerts, dead-letter queues, replay runbooks
Contract change Coordinated with known callers Schema is public; unknown consumers break silently
Failure mode Cascades and latency stacking Stuck consumers, backlog, late inconsistency
Testing Unit and contract tests per call Needs end-to-end flow tests and replay scenarios
Cross-team coordination Every new consumer touches the producer New consumers subscribe without producer changes

Synchronous calls save time because one request has one traceable path, and a failing test points at one hop. That cost grows with chain depth.

Illustrative estimate (assumed numbers, not a benchmark): chain availability = product of hop availabilities, and chain latency ≈ sum of hop latencies. With 4 hops at 99.9% and 40 ms each, you get 0.999⁴ ≈ 99.6% availability and about 160 ms before tail effects.

Events save time on coordination. A team can add a consumer without touching the producer. The price is that every field in the event becomes a long-lived contract.

Debugging time drops in both styles when every flow is documented as a list of who emits and who reacts, and when trace context propagates across the broker as well as across HTTP. Budget for that work up front, because adding it during an incident is expensive.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to event driven vs synchronous microservices, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • The dependency graph answers "what does this touch?" in seconds instead of a manual audit.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • You can size a change's blast radius before merging, avoiding surprise cross-team firefighting.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.