What does it mean to scale a monolith horizontally, and why do it?

To scale a monolith horizontally is to run several identical copies of the same application, each a full process with the same code and configuration, behind a load balancer that spreads requests across them. Nothing about the codebase's structure changes. Each replica can still serve every route.

The engineering work lies in finding every place the process quietly assumes it is the only copy. That assumption hides in session data held in memory, files written to local disk, in-process caches that drift apart, and scheduled jobs that fire once per replica instead of once per cluster. A stateless app, or one whose state lives in shared backing services, runs correctly no matter how many copies exist. Getting there is mostly an audit and a set of targeted fixes. It is not a redesign.

Teams do this for three main reasons:

  • Availability. With one instance, a crash or a bad host takes the whole service down. With several, the load balancer routes around the failed one.
  • Zero-downtime deploys. Replicas can be replaced one at a time while the others keep serving traffic.
  • Absorbing load. Adding replicas raises throughput without waiting on a bigger machine.

This approach suits teams with a working monolith that need more capacity or resilience. They can keep their codebase intact. Splitting the monolith into services means redrawing data ownership, adding network calls and inter-service contracts, and changing how the system is deployed. Making it replica-safe touches a bounded set of code paths and keeps one deployable unit, so the effort usually goes into fixes rather than restructuring.

                       +-----------------+
                       |     Clients     |
                       +--------+--------+
                                |
                                v
                       +-----------------+
                       |  Load balancer  |
                       +--------+--------+
                                |
          +---------------------+---------------------+
          |                     |                     |
          v                     v                     v
 +-----------------+   +-----------------+   +-----------------+
 |   Replica 1     |   |   Replica 2     |   |   Replica 3     |
 |  same code,     |   |  same code,     |   |  same code,     |
 |  same config,   |   |  same config,   |   |  same config,   |
 |  all routes     |   |  all routes     |   |  all routes     |
 +--------+--------+   +--------+--------+   +--------+--------+
          |                     |                     |
          +---------------------+---------------------+
                                |
                                v
 +------------------------------------------------------------------+
 |  Shared backing services: state lives here, not in the replicas  |
 |  +----------+  +---------------+  +-------------+  +-----------+ |
 |  | Database |  | Session store |  | Shared files|  | Job lock /| |
 |  |          |  | / cache       |  | (not local  |  | scheduler | |
 |  |          |  |               |  |  disk)      |  |           | |
 |  +----------+  +---------------+  +-------------+  +-----------+ |
 +------------------------------------------------------------------+

Identical monolith replicas sit behind a load balancer and keep sessions, cache, files and job coordination in shared backing services, so any replica can serve any request and the cluster runs correctly with any number of copies.

How does horizontal scaling of a monolith work?

A request's path through a horizontally scaled monolith has four parts.

  1. Routing. The client connects to the load balancer, not to any one replica. The load balancer keeps a list of backend replicas and picks one per request or per connection. Common strategies are round robin, which takes replicas in turn, and least connections, which picks the replica with the fewest open connections.
  2. Health checking. The load balancer, or the orchestrator that manages it, probes each replica on a schedule. A replica that fails its readiness check is taken out of rotation and gets no new traffic until it passes again. This is what lets the cluster route around a crashed or overloaded instance.
  3. Handling the request. The chosen replica runs the same code path a single instance would. Whenever the request needs state that must outlive it or be visible to other replicas, such as a session, an uploaded file, a cached permission, or a job lock, the replica reads or writes it in a shared backing service rather than in its own memory or disk. Because no replica holds anything the others need, the next request from the same user can go to a different replica and see the same state.
  4. Changing the replica count. To scale out, you start another copy with the same build and configuration. Once it passes readiness, the load balancer starts sending it traffic. To scale in or deploy, you drain a replica, which fails readiness and finishes in-flight work, then stop it. Rolling deploys repeat this one replica at a time, so the cluster stays up throughout.

The load balancer and the replicas are interchangeable parts. Correctness depends on the shared services and on the code never assuming it is alone. The rest of this article covers how to remove that assumption.

How do you audit a monolith for single-instance assumptions?

The audit starts by searching the codebase for anything that assumes "this process is the only one." The categories below cover most of it. Each one works with a single instance and fails once a load balancer spreads requests across replicas.

Hidden state Symptom under multiple replicas Target fix
In-memory sessions Users are logged out at random when a request lands on a replica that never saw their login External session store or signed client-side tokens
Local caches Replicas serve different values after a write. Invalidation reaches only one process Shared cache, or local cache with short TTLs and tolerance for staleness
Local filesystem writes An upload succeeds, then returns 404 from another replica. Files disappear on redeploy Object storage or a shared volume. Treat local disk as scratch only
In-process schedulers Every replica runs each job, so emails are sent N times and reports are duplicated Leader election or an external scheduler
In-memory rate limiters and counters Real limits become N times the configured value, and metrics undercount Counters in a shared store with atomic increments
WebSocket/SSE connection state Messages published on one replica never reach clients connected to another Pub/sub fan-out between replicas
Singleton background workers Queue consumers or pollers race each other and process items twice Idempotent handlers plus locking or a proper work queue

Grep for module-level variables, static maps, tmp paths, and timer or scheduler setup. Also check the configuration, because many frameworks keep sessions in memory and write to local disk by default.

The audit is never complete. Third-party libraries keep their own in-process caches, start background threads, or write to local paths without documenting it. Some of these assumptions appear only under real multi-replica traffic, so plan to keep finding them after rollout.

Should you use sticky sessions or move session state out of the process?

Sticky sessions (session affinity) make the load balancer route every request from a client to the same replica, usually via a cookie. They guarantee only routing. They do not keep the data that lives on that replica. When the replica restarts, is replaced during a deploy, or is removed by autoscaling, its in-memory sessions disappear and those users are logged out or lose work in progress. Affinity also skews load: long-lived or heavy clients pile onto whichever replicas they first reached, and newly added replicas receive only new sessions. It also hides bugs, because code that writes per-user state to local memory or disk appears to work until a client is routed somewhere else.

Sticky sessions are acceptable as a short-term step. They let you run several replicas while you move session state out of the process, provided you accept the logouts on each deploy.

The lasting fix is one of two designs:

Concern Shared session store Signed client-side token
Logout Delete the server-side record Clear the cookie; the token stays valid until expiry
Revocation Immediate Needs short expiry or a server-side denylist
Payload size Unbounded in practice Limited by cookie and header size limits
Per-request cost Network hop to the store Signature check only
Failure mode Store outage breaks every session; needs replication Key compromise or rotation affects all tokens

If you need immediate revocation, use the shared store and make it highly available. Signed tokens suit small, short-lived claims.

import json
import secrets

import redis
from itsdangerous import BadSignature, URLSafeTimedSerializer

store = redis.Redis(host="sessions.internal.example", port=6379)
SESSION_TTL_SECONDS = 1800

def create_session(user_id: int) -> str:
    sid = secrets.token_urlsafe(32)
    store.setex(f"session:{sid}", SESSION_TTL_SECONDS, json.dumps({"user_id": user_id}))
    return sid  # send to the client as the session cookie value

def load_session(sid: str):
    raw = store.get(f"session:{sid}")  # one network hop per request; Redis down = no sessions
    return json.loads(raw) if raw else None

def logout(sid: str) -> None:
    store.delete(f"session:{sid}")  # takes effect immediately on every replica

signer = URLSafeTimedSerializer("placeholder-load-secret-from-config", salt="session")
TOKEN_MAX_AGE_SECONDS = 900  # keep short, because there is no server-side record to delete

def issue_token(user_id: int) -> str:
    # Keep claims small: the whole payload travels in a cookie on every request.
    return signer.dumps({"user_id": user_id})

def verify_token(token: str):
    try:
        return signer.loads(token, max_age=TOKEN_MAX_AGE_SECONDS)  # signature check only
    except BadSignature:  # also covers SignatureExpired
        return None

A minimal Python sketch contrasting a Redis-backed session store, where logout deletes the server record, with an itsdangerous signed token, which any replica can verify but which cannot be revoked before it expires without extra state.

Moving files and caches to shared or explicitly local storage

Files: decide who owns each write

Any file written to local disk and read on a later request is a bug once a second replica exists, because the next request may land on a machine that never saw the write. User uploads, generated reports, and exports belong in object storage: every replica reads and writes the same bucket by key, and losing an instance loses nothing. A shared network volume also works and needs fewer code changes, but you inherit its locking, latency, and availability limits, so treat it as a stopgap rather than the target.

Temp files are fine when their whole lifetime fits inside one request on one instance, such as unpacking an archive before uploading the result. Write them to a per-request directory, delete them in a finally path, and never pass their paths to anything that might run elsewhere.

Caches: shared or deliberately local

Use a shared cache when replicas must agree: permission checks, feature flags that gate billing, anything where a stale read is visibly wrong. A per-instance in-memory cache suits data that tolerates staleness, like reference tables or rendered config. Give every entry a short TTL, because that TTL is your staleness bound.

Invalidation across replicas means either publishing an invalidation event that every instance subscribes to, or accepting the TTL as the only guarantee. When a hot shared entry expires, many replicas miss at once and stampede the database. Add jitter to TTLs, and let one caller recompute while the others briefly serve the old value.

import random
import threading
import time

BASE_TTL = 60.0  # seconds; this is the staleness bound for this instance
JITTER = 0.2     # +/-20% so entries loaded together don't all expire together

_cache = {}            # key -> (value, expires_at)
_refreshing = set()    # keys currently being reloaded by some caller
_lock = threading.Lock()

def _ttl():
    return BASE_TTL * random.uniform(1 - JITTER, 1 + JITTER)

def get(key, load):
    now = time.monotonic()
    with _lock:
        entry = _cache.get(key)
        if entry and entry[1] > now:
            return entry[0]            # fresh hit
        if entry and key in _refreshing:
            return entry[0]            # another caller is reloading: serve stale
        # Cold misses (no entry yet) may still load concurrently; only expired
        # entries are protected from the stampede.
        _refreshing.add(key)           # this caller owns the reload
    try:
        value = load(key)              # normally one caller per key hits the DB
        with _lock:
            _cache[key] = (value, time.monotonic() + _ttl())
        return value
    except Exception:
        if entry:
            return entry[0]            # DB failed: keep serving the old value
        raise
    finally:
        with _lock:
            _refreshing.discard(key)

A per-instance in-memory cache with a jittered TTL, where one caller reloads an expired key while concurrent callers keep getting the stale value instead of all hitting the database.

How does cron leader election make scheduled jobs run once across replicas?

When every replica runs the same in-process scheduler, a daily job fires once per replica. The goal is one runner per tick, and there are four common ways to choose it:

Mechanism How one runner is chosen What releases it
Database advisory lock Replica tries a non-blocking lock (e.g. PostgreSQL pg_try_advisory_lock) and skips if taken Connection close, so it does not work well with transaction-level connection poolers
Lease row with expiry Conditional UPDATE claims a row whose expires_at has passed; owner renews Expiry, judged by the database clock
Shared-store lock with TTL Set-if-not-exists key with an expiry TTL, or explicit delete by the owner
External scheduler Platform cron starts one job process; replicas never schedule Process exit; the scheduler itself may retry or double-fire

Any time-based lease can produce two leaders. Suppose a holder stalls in a GC pause, its clock runs fast, or a network partition cuts it off from the store. Its lease expires and another replica takes over, but the original holder resumes and keeps writing. Fencing tokens address this. Each grant carries a number that only increases, every write includes it, and the target rejects any token lower than the highest it has seen. This only helps if the downstream resource actually performs that check.

The more robust defence is to make running twice harmless. Derive an idempotency key from the job name and its scheduled slot, then insert it into a runs table with a unique constraint. The second insert fails and that run exits. Request handlers work the same way. The client sends an idempotency key, the server stores it with the response under a unique constraint, and a retry or duplicate delivery returns the stored result instead of repeating the side effect.

-- PostgreSQL. Each replica runs this at the tick; only one gets a row back.
CREATE TABLE job_runs (
  job_name   text        NOT NULL,
  slot       timestamptz NOT NULL,           -- the scheduled tick, not now()
  started_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (job_name, slot)               -- the idempotency key
);

INSERT INTO job_runs (job_name, slot)
VALUES ('daily-invoice-export', '2026-11-16 02:00:00+00')
ON CONFLICT DO NOTHING
RETURNING job_name;  -- zero rows: another replica or a retry already ran it, so exit

-- Lease row with a fencing token that only increases.
CREATE TABLE leases (
  name       text PRIMARY KEY,
  holder     text,
  token      bigint      NOT NULL DEFAULT 0,
  expires_at timestamptz NOT NULL
);
INSERT INTO leases (name, expires_at) VALUES ('scheduler', now())
ON CONFLICT DO NOTHING;  -- seed the row once

-- Claim only if expired; the database clock decides, not the replica's.
UPDATE leases
SET holder = 'replica-b', token = token + 1,
    expires_at = now() + interval '30 seconds'
WHERE name = 'scheduler' AND expires_at < now()
RETURNING token;  -- e.g. 42; send it with every downstream write

-- Downstream resource rejects writes from a stale leader (token 41 after a pause).
UPDATE export_state
SET last_slot = '2026-11-16 02:00:00+00', last_token = 42
WHERE id = 1 AND last_token <= 42;  -- 0 rows updated means: you are no longer leader

-- Request handlers: store the client's key with the response under a unique constraint.
CREATE TABLE idempotency_keys (
  key      text PRIMARY KEY,
  response jsonb
);

PostgreSQL showing a unique (job_name, slot) key that makes duplicate job runs exit, a lease row that issues increasing fencing tokens, a downstream write that rejects stale tokens, and an idempotency-key table for request handlers.

How do more replicas affect the database, startup and shutdown?

Every replica opens its own connection pool, so the database sees the product: per-replica pool size times replica count, plus workers, cron runners and admin tools. Illustrative estimate, with assumed inputs: a pool of 20 per replica across 12 replicas allows up to 240 connections before any background workers. Each server connection costs database memory and scheduling overhead. A database sized for a few hundred connections can therefore saturate while application CPU is still mostly idle, and adding replicas makes it worse. Shrink per-replica pools, and put a connection pooler in front of the database so many client connections share a small set of server connections. Transaction-level pooling breaks session-scoped features such as session advisory locks and some prepared-statement usage, so check what the app depends on first.

Run schema migrations once per deploy, as a separate step before the new replicas roll out. If every replica migrates at startup, they race to alter the same tables, and one failed migration can stop the whole fleet from booting. Write migrations so that both the old and new code work against the schema during the rollout.

Use two distinct health checks. Readiness means "route traffic to me" and passes only after the database is reachable and warm-up is done. Liveness means "restart me if I fail" and should check only the process itself. If liveness depends on the database, a brief outage restarts every replica at once.

On a termination signal, the replica should do these steps in order:

  1. Fail readiness.
  2. Stop accepting new requests.
  3. Let in-flight requests finish within a deadline.
  4. Release any held locks.
  5. Close its pools.

Read all configuration from environment variables rather than per-host files, so every replica is identical and replaceable.

What are the alternatives to scaling a monolith horizontally?

Horizontal scaling is one option among several. Most of them can be combined with it.

Alternative What it does Main trade-off
Vertical scaling Move the single instance to a machine with more CPU and memory No code changes, but there is a hardware ceiling, one instance is still a single point of failure, and resizing often means downtime
Offloading reads Add database read replicas and route read-only queries to them, or cache hot responses Helps when the database is the bottleneck, not the app tier. Reads from replicas can lag behind writes
Moving work off the request path Push slow tasks (emails, exports, image processing) onto a queue processed by separate workers Shortens request time and isolates heavy work, but adds a queue to operate and requires idempotent handlers
Serving static assets elsewhere Put static files behind a CDN or a dedicated static file server Removes easy load from the app, but does nothing for dynamic requests
Extracting one hot component Pull a single overloaded or stateful part into its own service, leaving the rest of the monolith in place Targets the real bottleneck, but that part now has network calls, its own deploys and its own data ownership
Full rewrite into services Split the whole application into independently deployed services Allows each part to scale on its own, but costs the most time and carries the highest risk of losing existing behavior

Vertical scaling is usually the first step because it costs nothing in code. Teams turn to horizontal scaling when they need more than one machine can give, or when they need availability that a single instance cannot provide. Extracting one component pairs well with horizontal scaling. It suits the workloads described in the next section that do not replicate cleanly.

How can scaling horizontally save engineering time?

The time savings come from what you do not have to build.

  • No rewrite. The domain logic, the data model and years of bug fixes stay where they are. You change how state is stored, not what the application does.
  • Existing tests remain valid. The behavior of each route is unchanged, so the current test suite still covers it. New tests are needed only for the replica-safety fixes themselves.
  • Fixes ship incrementally. Each item from the audit, such as moving sessions, moving uploads or adding a job lock, is a separate, small change that can be reviewed, deployed and rolled back by itself. You can run multiple replicas with sticky sessions early and remove the stickiness once the fixes land.
  • One build and one deploy. There is still one artifact, one pipeline and one version to reason about. You avoid coordinating releases across services, versioning internal APIs, and debugging failures that span network calls between services.
  • The fixes pay off beyond scaling. External sessions survive deploys, object storage survives instance loss, and idempotent jobs survive retries. These remove classes of incidents that cost on-call time even with a single instance.
  • Less time spent on capacity emergencies. Once the app is replica-safe, adding capacity is a configuration change rather than a migration to a bigger machine.

The work does not disappear. The audit, the new backing services and the database connection limits all need attention. But the effort is bounded by the number of single-instance assumptions in the code, which the audit makes visible up front. A rewrite's effort is bounded by the size of the whole system.

When should you avoid this approach, and how do you verify it is safe?

Some workloads resist this pattern. A job that builds a large in-memory structure, such as a full graph or a model loaded into RAM, does not split cleanly across replicas. Each copy either repeats the cost or needs coordination it was never designed for. Long-lived connection fan-out, such as WebSockets broadcasting to every subscriber, breaks once subscribers spread across instances, because a message published on one replica never reaches clients held by another. Strict ordering requirements, such as processing a ledger's events in sequence, conflict with parallel consumers by definition. These are usually better extracted into a separate service or kept on a single-instance worker, while the rest of the monolith scales out.

The cost is real but bounded. You add dependencies like a shared session store, object storage and a lock mechanism. Each one needs monitoring, backups and an on-call answer for when it is unavailable. That is still incremental work on code you already understand, which is far less risky than a rewrite that has to reproduce years of edge-case behavior.

Verifying it

Run two or three replicas in staging with stickiness disabled, so consecutive requests land on different instances. Then kill instances in the middle of requests and in the middle of scheduled jobs. Afterward, check three things:

  • whether any users were logged out,
  • whether any job ran twice or not at all,
  • whether replicas served stale cache entries after writes.

Roll out to production gradually, adding one replica at a time, and watch error rates and job logs closely before adding the next.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to scale monolith horizontally, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.