Choreography vs Orchestration: Where Saga State and Failures Live

What choreography and orchestration actually are

A saga is a long-running business transaction split into local transactions, one per service, where each step commits independently and a failure is undone by compensating actions rather than a distributed rollback. Choreography and orchestration are two ways to coordinate those steps. They differ in who decides what happens next.

Event choreography

In choreography, no component owns the workflow. The order service commits an order and publishes OrderPlaced. The payment service subscribes, charges the card and publishes PaymentCaptured or PaymentFailed. Inventory reacts to PaymentCaptured, and so on. Each service knows only which events it consumes and which it emits. The overall flow exists only in the combined subscriptions of every participant.

Workflow orchestration

In orchestration, a coordinator owns the flow. It calls payment, waits for the result, then calls inventory. It records each transition in durable storage, so it can resume after a crash and knows which compensations to run if a later step fails. Participants expose commands and do not need to know about each other.

Why the choice matters

The choice decides where workflow state and failure logic live. Under orchestration, "where is this order right now?" is a lookup against one record. Under choreography, the state is spread across service databases and broker offsets. Answering the same question means correlating events after the fact, and timeout and compensation rules end up spread across every consumer.

Copy CHOREOGRAPHY  (no owner: the flow is the sum of all subscriptions)

 +---------+        +---------+        +-----------+
 |  Order  |        | Payment |        | Inventory |
 | own DB  |        | own DB  |        |  own DB   |
 +---------+        +---------+        +-----------+
   |    ^             |    ^             |    ^
   v    |             v    |             v    |
 +-------------------------------------------------+
 |   Event broker: topics + consumer offsets       |
 +-------------------------------------------------+
   v = publish   ^ = subscribe
   OrderPlaced -> Payment;  PaymentCaptured -> Inventory
   "Where is order 42?" -> correlate events across DBs + offsets
   Timeouts and compensations: duplicated in every consumer

 ORCHESTRATION  (coordinator owns the flow and persists process state)

        +-----------------------------+      +-----------------------+
        |      Saga orchestrator      |----->| Workflow state store  |
        |  call step, await result,   |<-----| order 42: PAID,       |
        |  compensate on failure      |      | next: RESERVE_STOCK   |
        +-----------------------------+      +-----------------------+
           |     ^             |     ^
       cmd v     | reply   cmd v     | reply
        +---------+          +-----------+
        | Payment |          | Inventory |
        +---------+          +-----------+
   "Where is order 42?" -> read one record in the state store

In choreography the saga's state and failure rules are scattered across services and the broker, while in orchestration a coordinator keeps them in one durable record.

One order saga implemented both ways

To make the comparison concrete, take one order: reserve inventory, charge payment, schedule shipment. If payment fails, release the reservation and cancel the order.

Choreographed version

Each service reacts to the previous service's event. The Order service publishes OrderPlaced { orderId, items, customerId }. Inventory consumes it, reserves stock and publishes InventoryReserved { orderId, reservationId }. Payment consumes that and publishes either PaymentCaptured { orderId, paymentId } or PaymentFailed { orderId, reason }. Shipping listens for PaymentCaptured.

The compensation path works the same way. Inventory subscribes to PaymentFailed and releases the reservation. Order also subscribes to it and marks the order cancelled. No component holds the whole sequence. The process definition exists only as the combined subscription lists of four services, so you can only reconstruct it by reading all four codebases.

Orchestrated version

An order workflow sends commands and waits for replies:

• Send ReserveInventory { orderId, items }.
• Send ChargePayment { orderId, amount }.
• On success, send ScheduleShipment { orderId, address }.
• On failure, send ReleaseInventory { reservationId }, then CancelOrder { orderId, reason }.

The services become simpler. They expose operations and reply with results. They no longer decide what happens next. The process definition, including the compensation order, lives in one file owned by one team.

The events and commands carry nearly identical data. The difference is where the decisions are made. In choreography, every service knows its position in the sequence. In orchestration, only the coordinator does. That location determines who owns a failure when one occurs.

Copy// Orchestrated order saga: the full process definition, including compensation,
// lives in this one function. Services only execute commands and reply.
type Order = { orderId: string; items: { sku: string; qty: number }[]; amount: number; address: string };
type Reply = { ok: true; data: Record<string, string> } | { ok: false; reason: string };

// Transport is abstracted: request/reply over a queue or RPC.
// Production coordinators must persist saga state after each step so a crash resumes
// instead of restarting. Every command must be idempotent (keyed by orderId) for safe retries.
declare function send(service: string, command: { type: string } & object): Promise<Reply>;

export async function placeOrderSaga(order: Order): Promise<void> {
  const { orderId } = order;

  const reserved = await send("inventory", { type: "ReserveInventory", orderId, items: order.items });
  if (!reserved.ok) {
    // Nothing to undo yet; just cancel the order.
    await send("orders", { type: "CancelOrder", orderId, reason: reserved.reason });
    return;
  }
  const reservationId = reserved.data.reservationId;

  const charged = await send("payment", { type: "ChargePayment", orderId, amount: order.amount });
  if (!charged.ok) {
    // Compensate completed steps in reverse order. In the choreographed version,
    // this same logic is split across Inventory's and Order's PaymentFailed handlers.
    await send("inventory", { type: "ReleaseInventory", reservationId });
    await send("orders", { type: "CancelOrder", orderId, reason: charged.reason });
    return;
  }

  await send("shipping", { type: "ScheduleShipment", orderId, address: order.address });
}

A minimal orchestrator for the order saga. The step sequence and the payment-failure compensation (release inventory, then cancel order) sit in one function instead of being spread across service subscriptions.

Where workflow state lives and how to answer status queries

In choreography, no service owns the answer to "where is order 4812?" The state is implied by the combination of the order row, the payment record, the inventory reservation and the events in the log. To answer a status query you either query each service and reconcile the results, or you replay events. A stuck process is hard to spot because it does not fail. It just stops, and no component is waiting for the event that never arrives.

An orchestrator persists the workflow instance explicitly: current step, inputs, retry counts and timers. Status becomes a single lookup. A stuck instance shows up as a step that has been waiting past its deadline, and the orchestrator can alert on it because it knows what it expected next.

That explicit state has a cost in versioning. Long-running instances started under the old definition are still in flight when you deploy a new one. You need to either run both versions side by side until old instances drain, or migrate persisted state deliberately. Choreography hides this problem rather than removing it. Changing an event contract mid-flight breaks consumers silently.

Making choreography operable

• Correlation IDs: Propagate one saga ID through every event header and synchronous call, and log it everywhere.
• Distributed tracing: Carry trace context across the broker so asynchronous hops appear in one trace.
• Process view: Run a consumer that subscribes to all saga events and maintains a per-instance projection with timeouts. This effectively rebuilds read-only orchestrator state for querying and alerting.

Copyimport sqlite3, time

# Read-only process view built from saga events. It answers status queries and
# finds stuck sagas without asking each service.
db = sqlite3.connect("saga_view.db")
db.execute("""CREATE TABLE IF NOT EXISTS saga_view (
  saga_id TEXT PRIMARY KEY, last_event TEXT, expected_next TEXT,
  deadline REAL, updated_at REAL)""")

# Encodes what the choreography implies: after each event, which event should
# follow and how long (seconds) we tolerate waiting. None marks a terminal state.
NEXT = {
    "OrderPlaced":       ("PaymentCaptured", 300),
    "PaymentCaptured":   ("InventoryReserved", 120),
    "InventoryReserved": ("OrderConfirmed", 60),
    "OrderConfirmed":    (None, None),
    "PaymentFailed":     (None, None),
}

def on_event(event):
    # Called by the broker consumer for every saga event. Assumes per-saga
    # ordering, e.g. topic partitioned by saga_id taken from the event header.
    saga_id = event["headers"]["saga_id"]
    expected, timeout = NEXT.get(event["type"], (None, None))
    now = time.time()
    deadline = now + timeout if timeout else None
    db.execute("""INSERT INTO saga_view VALUES (?, ?, ?, ?, ?)
      ON CONFLICT(saga_id) DO UPDATE SET last_event = excluded.last_event,
        expected_next = excluded.expected_next, deadline = excluded.deadline,
        updated_at = excluded.updated_at""",
      (saga_id, event["type"], expected, deadline, now))
    db.commit()

def status(saga_id):
    # "Where is order 4812?" becomes a single lookup.
    return db.execute("SELECT * FROM saga_view WHERE saga_id = ?", (saga_id,)).fetchone()

def stuck():
    # A saga is stuck when the event it was waiting for never arrived by its deadline.
    return db.execute("""SELECT saga_id, last_event, expected_next FROM saga_view
      WHERE deadline IS NOT NULL AND deadline < ?""", (time.time(),)).fetchall()

A minimal process-view consumer that projects choreographed saga events into a per-instance table, keyed by the propagated saga ID, so you can query status with one lookup and alert on sagas waiting past their deadline.

Failure handling: timeouts, retries, idempotency and compensations

In choreography, no service owns a deadline. If Payment never emits PaymentSucceeded, Inventory keeps waiting, so you need a separate timer service or the process view above to notice. An orchestrator owns each step's deadline. It schedules the timeout, retries with backoff, and decides when to give up and compensate. In both models delivery is at-least-once, so every consumer must be idempotent. Record the message ID in a processed-messages table in the same transaction as the business write, and skip any ID already recorded.

Compensations are not rollbacks. By the time you compensate, other transactions may already have seen the reservation or the captured funds. ReleaseStock and RefundPayment are new forward actions. They can fail on their own, so they must be idempotent and retried until they succeed.

Some actions cannot be undone. Order the saga so these run after the pivot step, the point beyond which the saga only moves forward. Send the confirmation email only after payment is captured. For external payments, authorize early and capture late. Compensating then means voiding an authorization instead of issuing a refund the customer can see.

Both models share one hazard: the database write and the message publish go to two separate systems. The transactional outbox closes that gap. Insert the event into an outbox table (id, aggregate_id, type, payload, created_at, published_at) in the same transaction as the state change. A relay reads unpublished rows in order, publishes them, then sets published_at. If the relay crashes between publishing and updating, the event is sent twice, and idempotent consumers absorb the duplicate.

Copy-- PostgreSQL. Outbox: events are written in the same transaction as the state change.
CREATE TABLE outbox (
  id           uuid PRIMARY KEY,
  aggregate_id uuid NOT NULL,
  type         text NOT NULL,
  payload      jsonb NOT NULL,
  created_at   timestamptz NOT NULL DEFAULT now(),
  published_at timestamptz              -- NULL means the relay has not sent it yet
);
CREATE INDEX outbox_unpublished ON outbox (created_at) WHERE published_at IS NULL;

-- Consumer side: remembers which message IDs have already been applied.
CREATE TABLE processed_messages (
  message_id   uuid PRIMARY KEY,
  processed_at timestamptz NOT NULL DEFAULT now()
);

-- Producer: either both rows commit or neither does. There is no dual write.
BEGIN;
UPDATE orders SET status = 'PAYMENT_PENDING' WHERE id = '7f3c9a2e-0000-4000-8000-000000000001';
INSERT INTO outbox (id, aggregate_id, type, payload)
VALUES ('b1e2d3c4-0000-4000-8000-000000000002', '7f3c9a2e-0000-4000-8000-000000000001',
        'OrderPlaced', '{"orderId":"7f3c9a2e-0000-4000-8000-000000000001","amountCents":4999}');
COMMIT;

-- Relay: claim a batch in order. SKIP LOCKED lets several relays run in parallel,
-- but then strict ordering is lost. Run a single relay if consumers depend on order.
BEGIN;
SELECT id, type, payload FROM outbox
WHERE published_at IS NULL ORDER BY created_at LIMIT 100
FOR UPDATE SKIP LOCKED;
-- (application publishes each row to the broker here)
-- A crash after publishing but before this UPDATE causes a resend on the next run.
UPDATE outbox SET published_at = now() WHERE id = ANY($1);  -- $1 = IDs that were published
COMMIT;

-- Consumer: dedupe and apply the business change in one transaction.
-- If the INSERT affects 0 rows, the message is a duplicate: skip the work and commit.
BEGIN;
INSERT INTO processed_messages (message_id) VALUES ($1) ON CONFLICT DO NOTHING;
COMMIT;

A minimal PostgreSQL transactional outbox: the producer writes its state change and its event atomically, a relay publishes unpublished rows and marks them sent, and consumers use a processed-messages table to absorb the duplicates that at-least-once delivery produces.

Coupling, change cost and engineering time

Neither model removes coupling. Each one moves it to a different place.

In choreography, services couple to event schemas and to a sequence that nobody wrote down. Renaming a field, or splitting PaymentCaptured into two events, is a contract change for every subscriber, including ones you don't know about. Schema evolution therefore becomes a hidden API that has no owner. Reordering a step, such as reserving stock before charging instead of after, means changing which event each service listens to. That change usually spans several teams' repositories and release schedules. As flows grow, services start reacting to each other's reactions. This produces cyclic dependencies and emergent behavior that no single diagram describes.

Orchestration couples the coordinator to service APIs instead. Reordering becomes a change in one repository, but the orchestrator is now stateful infrastructure. It needs durable storage, upgrades and an on-call owner, and while it is down, no saga advances. Teams also tend to push business logic into it until it becomes a "god service" that every change has to queue behind. Workflow engines add a learning curve as well. Replay-based engines require deterministic workflow code, so clock reads, random values and I/O must go through engine APIs. Changing code that in-flight workflows are still running requires explicit versioning.

When to use each, when to avoid them, and the alternatives

Pick the approach by the shape of the process, not by preference. The heuristics below hold for most saga designs.

Choreography fits reactions that are loosely related: send a notification, update analytics, invalidate a cache. If a consumer fails, the business outcome doesn't change. Avoid it when a step must undo earlier ones. Compensation logic spread across consumers is the point where cycles and emergent behavior appear.

Orchestration fits long-running processes that change often, because the flow lives in one place you can version and inspect. Avoid it for fire-and-forget fan-out. An orchestrator that does nothing but publish events adds a stateful dependency without giving you any control.

There are alternatives to both:

• Hybrid. Orchestrate inside a bounded context, then broadcast the outcome, such as OrderConfirmed, as an event for other contexts to react to. This is the most common healthy end state.
• Synchronous call chain. Use this when every step is fast, owned by one team, and safe to fail the whole request.
• One service, one transaction. If the data can live in a single database, an ACID transaction removes the need for a saga entirely.

On ownership, the team that owns the business outcome owns the orchestrator. Downstream teams own their own event consumers. Platform teams provide the workflow engine and the broker, not the workflows.

Migrating a tangled choreographed flow to orchestration incrementally

Rewriting a choreographed flow in one release fails because nobody has a full map of who reacts to what. Build the map first, then move control over to it a step at a time.

Phase 1: observe only

Deploy a coordinator that subscribes to the existing events and records process state keyed by order ID. It issues no commands. This is low risk: if it is wrong, nothing breaks except its own view. Within days it answers "where is order 4411?" and flags instances stuck past a deadline, which is often the operational win teams wanted in the first place. Compare its derived state against reality. Mismatches expose the undocumented subscribers and implicit ordering you would otherwise find during cutover.

Phase 2: take over one step

Pick a single transition, such as reserving inventory after payment succeeds. The coordinator starts sending ReserveInventory as a command, and you remove inventory's subscription to PaymentSucceeded. Services keep publishing outcome events like InventoryReserved, because shipping analytics and notifications in other bounded contexts still depend on them. Repeat for each step.

Cutover risks

During each switch there is a window when both the old subscription and the new command can fire. Before flipping:

• Make the receiving handler deduplicate on a business key such as order ID plus step, not on message ID. The two paths produce different messages.
• Gate the new path behind a per-instance flag so in-flight sagas finish on the old path.
• Alert on any instance that receives both triggers. That count should be zero before you delete the old consumer.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to choreography vs orchestration, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• The dependency graph answers "what does this touch?" in seconds instead of a manual audit.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• You can size a change's blast radius before merging, avoiding surprise cross-team firefighting.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.