What does scaling a monolith horizontally require, and how do you find hidden per-instance state?
Scaling a monolith horizontally means running several identical copies of the same application behind a load balancer, where any request can land on any instance and still produce the correct result. Teams do this for three reasons:
- Capacity: add nodes instead of buying a bigger one.
- Redundancy: losing one instance does not take the site down.
- Zero-downtime deploys: instances can be replaced one at a time.
It suits teams whose codebase is healthy but whose single server has become a bottleneck or a single point of failure.
The blocker is almost always state that quietly lives inside one process or on one disk. Inventory it before adding nodes:
| State | Where it hides | Symptom with two instances |
|---|---|---|
| Sessions | In-process session store | Random logouts |
| Uploads, reports, temp files | Local disk | 404 on download from the other node |
| Caches | In-memory maps | Stale or inconsistent reads |
| Rate-limit counters | Process memory | Limits effectively multiplied by node count |
| WebSocket connections | Open sockets per node | Messages miss users connected elsewhere |
| Cron and scheduled jobs | Scheduler started at boot | Jobs run once per instance |
| Locks | Mutexes, file locks | Double-processing of the same record |
The audit
Code review misses things, so test the behaviour directly:
- Run two instances in staging behind a plain round-robin balancer with stickiness explicitly off. The config example below shows a minimal setup.
- Exercise every critical flow end to end: login, checkout, upload then download, report generation, password reset.
Alternating requests between nodes forces each hidden dependency to fail visibly. Each failure you find maps to a row in the table above.
events {}
http {
upstream app_staging {
# Two identical instances of the monolith (placeholder hosts/ports).
server app-1.staging.internal:8080;
server app-2.staging.internal:8080;
}
server {
listen 80;
server_name staging.example.com;
# Match the app's real upload limit so upload-then-download is tested.
client_max_body_size 50m;
location / {
proxy_pass
# Preserve client info so the app sees what it will in production.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Expose which node served each request, so a failure can be
# traced to the instance that lacked the state.
add_header X-Upstream $upstream_addr always;
}
}
}
A minimal nginx config that alternates requests between two staging instances with no session affinity and labels each response with the node that served it, so hidden per-instance state shows up as visible failures.
Who should scale a monolith horizontally, and who should not?
Horizontal scaling fits a specific situation. It is a good choice when most of the following are true:
- The monolith works and you intend to keep it. The code is maintainable, and splitting it into services would cost more than it returns. Horizontal scaling keeps one codebase and one deploy artifact.
- One server is the limit. CPU or memory on a single instance is close to saturated at peak, or the next size up is expensive or unavailable.
- Downtime is costly. Deploys that need a maintenance window, or a single host whose failure takes the product offline, are no longer acceptable.
- The team can run a few shared components. You will need a session store or a key-management process, object storage, and shared coordination for jobs. Someone has to own provisioning, monitoring and backups for them.
- The database can absorb more connections. Every new instance opens its own pool against the same database, so the database must have connection headroom. Otherwise you need a pooler in front of it.
It is a poor fit, or should wait, when:
- One instance handles peak load with room to spare, and brief downtime during deploys is acceptable.
- The application depends on an embedded database file on local disk. SQLite is a common example. Copies of the app cannot share that file safely, so the data has to move to a networked database first.
- The bottleneck is the database rather than the app servers. More app instances add load to the database and do nothing to relieve it.
- No one on the team can take on the extra infrastructure. In that case a larger instance with a warm standby is the safer choice. The alternatives section below covers that option.
How does horizontal scaling of a monolith work?
The mechanism is simple. What makes it work is the discipline about where state lives.
- A load balancer is the single entry point. Clients connect to the balancer, not to an instance. The balancer keeps a list of healthy backends and forwards each request to one of them using round-robin, least-connections or a similar method.
- Every instance runs the same build with the same config. Instances are interchangeable. None of them owns a user, a file or a job. They load secrets, signing keys and feature flags from one shared source.
- State lives outside the instances. Durable data stays in the database. Sessions move to a shared store or into signed cookies. Uploads go to shared storage. Caches, rate-limit counters and real-time fan-out use shared services. An instance can then be created or destroyed without losing anything a user depends on.
- Coordination replaces "only one process" assumptions. Work that must happen once, such as scheduled jobs or processing a given record, is guarded by a database lock, a conditional update or a dedicated worker. It no longer relies on there being one process.
- Health checks control membership. The balancer probes each instance's readiness endpoint. A failing instance stops receiving traffic. A new instance receives traffic once it reports ready. Adding capacity therefore means starting more instances and registering them with the balancer.
- Deploys roll through the fleet. The balancer drains an instance and the instance is replaced with the new version. The process repeats until every node runs the new code. This only works if old and new versions can read each other's sessions and files, which is why later sections focus on serialization formats and migration order.
The rest of this article covers each step in that order: finding the state, moving sessions, moving uploads, deciding whether to use stickiness, fixing the remaining shared-state traps, and operating the result.
How do stateless sessions work with a shared session store like Redis or your database?
The most common fix is to keep sessions server-side but move them out of process memory into a store every instance can reach. The cookie then carries only an opaque session ID, so any node can serve any request.
Expiry. Set a TTL on every session key, and keep it in line with the cookie's lifetime. If the cookie has Max-Age=1800, write the key with the same TTL. In Redis that is SET session:<id> <payload> EX 1800. If the store keeps keys longer than the cookie, you collect orphaned sessions. If it expires them sooner, users get logged out while still holding a valid cookie. For sliding sessions, refresh both the TTL and the cookie on activity. In a relational database there is no native TTL, so you store an expires_at column, filter on it at read time, and run a cleanup job.
Serialization during rolling deploys. Old and new code read the same session records at the same time. Use an explicit format such as JSON with a version field, add fields as optional, and hold off on removing fields until the old code is fully gone. Language-native serialization can make one version's sessions unreadable to the other.
Store outages. If the store is down, authenticated requests fail. Decide in advance whether to return errors or treat users as logged out, and test that path.
Trade-offs.
- Every authenticated request adds a network round trip to the store.
- The store becomes a new single point of failure and needs its own replication and failover plan.
- Redis running without persistence loses every session on restart, which logs out all users at once.
import json
import secrets
import redis
SESSION_TTL = 1800 # seconds; single source of truth for store TTL and cookie Max-Age
r = redis.Redis(host="sessions.internal.example", port=6379, decode_responses=True)
def create_session(user_id: int) -> str:
session_id = secrets.token_urlsafe(32) # opaque ID; the cookie carries nothing else
# Explicit, versioned JSON so old and new code can both read it during a rolling deploy
payload = json.dumps({"v": 2, "user_id": user_id, "roles": ["member"]})
# EX sets the key TTL to match the cookie lifetime exactly
r.set(f"session:{session_id}", payload, ex=SESSION_TTL)
# Same lifetime on the cookie, so neither side outlives the other
return (f"sid={session_id}; Max-Age={SESSION_TTL}; Path=/; "
"HttpOnly; Secure; SameSite=Lax")
def load_session(session_id: str) -> dict | None:
key = f"session:{session_id}"
try:
raw = r.get(key)
if raw is None:
return None # expired or never existed
r.expire(key, SESSION_TTL) # sliding session: caller must also re-send the cookie
except redis.exceptions.ConnectionError:
# Policy decided in advance: store outage means "treat as logged out"
return None
data = json.loads(raw)
# Fields added in v2 are optional so v1 records still load
data.setdefault("roles", [])
return data
A Python session store using redis-py that writes each session key with an expiry equal to the cookie's Max-Age. It stores a versioned JSON payload so old and new code can read it during rolling deploys, and it treats a Redis outage as logged out.
When should you use signed client-side session cookies instead of a session store?
The alternative removes the store entirely. The application serialises the session into a cookie and signs it with an HMAC so that tampering can be detected. It can also encrypt the cookie if the contents should not be readable by the client. Any instance holding the key can validate any request, so there is nothing shared to replicate or fail over.
The trade-offs are concrete:
- Size. Browsers generally cap a single cookie at about 4 KB, and the cookie is sent on every request to its path. Keep it to an identifier, roles and an expiry. Do not put carts or cached profile data in it.
- Key distribution. Every node needs the same secret. Load it from your secrets manager at startup, not from the image. A node with a different key will reject sessions the others issued, and users will see intermittent logouts.
- Rotation. Configure a list of keys. New cookies are signed with the newest key, and validation accepts any key in the list during an overlap window that lasts at least as long as the session lifetime. After the window closes, remove the old key.
- Revocation. The server can't delete a cookie it no longer sees. To support "log out everywhere" or banning a user, you need a server-side check, such as a per-user session version or a denylist. That check reintroduces a lookup, so keep it small and cacheable.
- Key leakage. Anyone who holds the signing key can mint valid sessions for any user. Forged sessions keep working until you rotate the key, and rotation logs out all users.
| Shared store | Client-side signed cookie | Sticky sessions | |
|---|---|---|---|
| Revocation | Delete the record | Requires an extra server-side check | Delete in-process, if you can reach the right node |
| Failure behaviour | Store outage blocks all logins | Survives node loss; key leak is severe | Node loss drops its users' sessions |
| Operational cost | Run and replicate a store | Manage and rotate keys | Low until rebalancing |
| Deploy compatibility | Rolling deploys are safe | Safe if all nodes share the keys | Draining a node logs users out |
import base64, hashlib, hmac, json, os, time
KEYS = dict(k.split(":", 1) for k in os.environ["SESSION_KEYS"].split(","))
CURRENT_KID = next(iter(KEYS)) # sign new cookies with the newest key only
def _b64(b): return base64.urlsafe_b64encode(b).rstrip(b"=").decode()
def _unb64(s): return base64.urlsafe_b64decode(s + "=" * (-len(s) % 4))
def _sig(kid, body):
mac = hmac.new(KEYS[kid].encode(), f"{kid}.{body}".encode(), hashlib.sha256)
return _b64(mac.digest())
def issue(uid, roles, ver, ttl=3600):
# Keep it small (4 KB cookie cap): id, roles, session version, expiry.
claims = {"uid": uid, "roles": roles, "ver": ver, "exp": int(time.time()) + ttl}
body = _b64(json.dumps(claims, separators=(",", ":")).encode())
return f"{CURRENT_KID}.{body}.{_sig(CURRENT_KID, body)}"
def verify(cookie, current_version_for):
try:
kid, body, sig = cookie.split(".")
except ValueError:
return None
if kid not in KEYS: # old key removed once the overlap window closed
return None
if not hmac.compare_digest(sig, _sig(kid, body)): # constant-time compare
return None
claims = json.loads(_unb64(body))
if claims["exp"] < time.time():
return None
# Revocation: a per-user version bumped on "log out everywhere" or a ban.
# This is the one server-side lookup; keep it small and cacheable.
if claims["ver"] != current_version_for(claims["uid"]):
return None
return claims
A minimal stdlib HMAC-signed session cookie (signed, not encrypted) that tags each cookie with a key ID, accepts every key in the configured list during rotation, and checks a per-user session version so sessions can be revoked server-side.
How do you move file uploads to shared file storage?
Any code that writes to a local path breaks once a second node exists, because the upload lands on one disk and the next request may hit another. There are two replacement targets.
| Network file system (NFS) | Object storage (S3-compatible API) | |
|---|---|---|
| Code changes | Minimal, same file APIs | Replace file I/O with an SDK client |
| Latency | Every open, stat and write crosses the network | Per-request HTTP overhead |
| Locking | Advisory locks unreliable across clients | No locks, whole-object writes |
| Outage behaviour | Hung mounts can block worker processes | Errors returned, app can degrade |
| Permissions | Unix users and groups | Bucket policies and credentials |
NFS looks cheaper but fails worse. A stalled mount freezes threads instead of raising an error.
You can proxy upload bytes through the app, which keeps validation inline but ties up workers for the whole transfer. The alternative is to have the app issue a pre-signed URL. This is a short-lived, signed PUT for a single key, which you can constrain by content type and length. The client then uploads directly to storage. The trade-off is that type checks, size verification and malware scanning become an async step, so unapproved files exist. Write them to a quarantine prefix and promote them only after scanning.
Direct uploads also create a race. The client's follow-up "attach" request can reach any node before the upload has finished. Confirm the object exists with a HEAD request, or track upload state in the database.
To migrate, copy existing files to storage. Then deploy a read path that checks storage first and falls back to local disk. During the rollout, old nodes still write locally, so rerun the copy after every node runs the new code.
import uuid
import boto3
from botocore.exceptions import ClientError
s3 = boto3.client("s3", region_name="eu-west-1")
BUCKET = "example-app-uploads"
ALLOWED_TYPES = {"image/png", "image/jpeg", "application/pdf"}
def create_upload(user_id: str, content_type: str) -> dict:
if content_type not in ALLOWED_TYPES:
raise ValueError("unsupported content type")
# Unscanned files go under quarantine/ and are promoted only after the async scan.
key = f"quarantine/{user_id}/{uuid.uuid4()}"
url = s3.generate_presigned_url(
"put_object",
# ContentType is signed, so the client must send this exact header.
Params={"Bucket": BUCKET, "Key": key, "ContentType": content_type},
ExpiresIn=300, # short-lived: valid for 5 minutes
)
# Persist the key with state 'pending' so any node can find it later.
return {"key": key, "url": url}
def upload_finished(key: str) -> bool:
# The 'attach' request may hit a different node before the PUT completes.
# A HEAD request checks shared storage instead of any local state.
try:
head = s3.head_object(Bucket=BUCKET, Key=key)
except ClientError as err:
if err.response["Error"]["Code"] in ("404", "NoSuchKey", "NotFound"):
return False
raise
# The size is not enforced by this PUT URL, so check it here and again in the scanner.
return head["ContentLength"] <= 10 * 1024 * 1024
Using boto3, the app issues a short-lived pre-signed PUT URL into a quarantine prefix, then checks with a HEAD request that the object exists before handling the follow-up attach request on any node.
What do sticky sessions actually do, and when should you avoid relying on them?
Sticky sessions (session affinity) make the load balancer route a client back to the same instance it used before. There are two common forms. With cookie-based affinity, the load balancer either sets its own cookie naming the chosen backend or hashes an existing application cookie. With source-IP affinity, it hashes the client's IP address to choose a backend. Neither changes your application. The balancer simply stops spreading one user's requests across nodes, so per-instance state appears to keep working.
That makes stickiness a reasonable short-term bridge. If sessions still live in process memory while you build the shared store, affinity lets you add a second node without logging users out on every request.
It does not remove the state problem. It only puts that state on a single node:
- Node loss or drain drops sessions. When an instance crashes, is drained for a deploy, or is scaled in, every session pinned to it is gone.
- Load distributes unevenly. Long-lived or heavy clients stay pinned. New nodes added by autoscaling receive only new sessions, so they take little of the existing load.
- IP affinity is distorted by NAT and proxies. An office or mobile carrier behind one address lands entirely on one node. A client whose IP changes mid-session loses its affinity.
Stickiness takes one setting to enable, and it hides real bugs, such as uploads written to local disk or caches that never invalidate. Teams often forget it is on until the first node failure or scale-in event. When you enable it, set a removal date and track it as an explicit task with an owner.
Which shared-state traps break after sessions and uploads are fixed?
Moving sessions and uploads off the node removes the obvious state. Running a second copy of the app usually exposes four more problems, because code written for one process assumes it is the only one.
| Trap | Why it breaks with N nodes | Fix |
|---|---|---|
| Scheduled jobs | Every node runs the in-process scheduler, so a nightly invoice job runs N times. | Leader election, a DB-backed lock, or a separate single worker process |
| Local in-memory cache | Node A updates a record. Node B keeps serving its cached copy until the TTL expires. | Broadcast invalidation messages, or move to a shared cache |
| Per-node rate limiting | Each node counts only its own requests, so the effective limit is the configured limit × instance count. | Shared counters in Redis or the database |
| WebSocket/SSE fan-out | An event raised on node A cannot reach clients connected to node B. | A pub/sub backplane that every node subscribes to |
If you want to keep the scheduler inside the app, take a database lock before doing any work. In PostgreSQL, call pg_try_advisory_xact_lock with a fixed integer key per job:
- One node gets
true, runs the job, and releases the lock when the transaction commits. - Every other node gets
falseand skips the run.
A conditional row update works on any SQL database. Run UPDATE job_runs SET last_run = now() WHERE name = 'invoices' AND last_run < now() - interval '1 day'. Only the node whose update affects exactly one row proceeds.
The two mechanisms behave differently on failure. The advisory lock is held for the length of the job and released automatically if the connection dies. The row update claims the run up front, so a crash partway through skips that day's run unless you record completion separately.
How do you deploy, operate and verify a horizontally scaled monolith?
- Health checks and connection draining. Expose a readiness check separate from liveness. On shutdown, fail readiness first so the load balancer stops sending new requests, then let in-flight work finish before the process exits. Set the drain timeout longer than your slowest expected upload. Otherwise every deploy truncates large multipart uploads.
- Identical config and secrets. Every node must load the session signing key, storage location and feature flags from one source. A single node with a stale signing key rejects cookies the others issued, and users see what looks like random logouts.
- Centralized logs with a request ID. Accept or generate an ID at the edge, pass it in a header, and include it in every log line and outbound call. Without it, one user's session spread across several nodes cannot be reconstructed.
- Connection budget. The total is pool size × instance count, plus any surge instances during deploys. Illustrative estimate, assumed inputs: a database limit of 200 connections with 20 reserved for admin and migrations leaves 180. Six instances × 20 connections each is 120. Two surge instances add 40, and background workers take 20, for 180 in total. A seventh instance exceeds the limit.
- Rolling-deploy compatibility. Old and new versions serve traffic at the same time. New code must read the old session shape and old file paths. Switch writes to the new format only after every node runs the new code.
- Verify in staging. Kill or drain one node while a user is logged in and an upload is in progress. Confirm the user stays logged in and the file remains reachable from other nodes. Then turn off stickiness and repeat the test.
How can horizontal scaling save engineering time?
The savings come from work that stops happening, not from faster code. Once instances are interchangeable, several recurring tasks either disappear or become routine:
- No maintenance windows for deploys. Rolling deploys replace instances one at a time behind the balancer. Nobody schedules off-hours releases, sends downtime notices, or waits for a quiet period to ship a fix.
- Node failures stop being incidents. When an instance dies, the balancer's health check removes it and the remaining nodes keep serving traffic. Replacing the node is routine work, not an emergency. With a single server, the same failure means someone gets paged, diagnoses the problem and runs a manual failover.
- Capacity changes become configuration. Adding headroom means starting another instance and registering it. You avoid resizing a host, which often requires a restart, and you avoid migrating to new hardware.
- Fewer hard-to-reproduce bugs. Without affinity, every request can land on any node. Hidden per-instance state therefore fails in staging during the audit, and intermittent "it works for some users" reports become less likely in production. The request ID in centralized logs shortens the remaining investigations because one user's path across nodes can be reconstructed.
- Safer, smaller releases. A bad build can be caught on the first replaced instance, then rolled back before the rest of the fleet is affected. That makes frequent small deploys practical, and small deploys are quicker to review and debug.
These savings have a price. The audit, the session and upload migrations, and the coordination fixes are up-front work. The new shared components also need ongoing upkeep. Whether the net result saves time depends on how often you deploy, how often nodes fail, and how much those events cost your team today. The illustrative calculation in the next section shows how to weigh the two sides with your own inputs.
What are the alternatives to scaling a monolith horizontally, and what does it cost?
Horizontal scaling is not the default answer. If one instance handles peak load with headroom, a larger single instance plus a warm standby is often simpler. You keep in-process sessions and local disk, and failover becomes a promotion step rather than a design constraint. The trade-off: failover drops sessions, interrupts in-flight uploads, and deploys still need a maintenance window.
Going stateless adds a session store, object storage, and possibly pub/sub and a distributed lock service. Each one needs provisioning, monitoring, backups and on-call knowledge. In return you get zero-downtime deploys, node failures that need no incident response, and no affinity bugs to debug later.
Illustrative estimate (assumed numbers, not a benchmark): Monthly net = (incidents avoided × hours per incident) + (deploy windows avoided × hours per window) − (new components × upkeep hours per component) = (1 × 6) + (4 × 1) − (3 × 2) = +4 hours/month
Plug in your own numbers. If the result is negative and stays negative as traffic grows, stay vertical.
| Traffic | Availability need | Team size | Recommended setup |
|---|---|---|---|
| Fits one instance with headroom | Brief downtime acceptable | Small | Vertical scaling with warm standby |
| Outgrowing one instance | Moderate; migration in progress | Small to medium | Sticky sessions as a temporary bridge |
| Needs several instances | No downtime on deploys or node loss | Can own extra infrastructure | Fully stateless sessions and shared storage |
Treat the sticky bridge as having an end date. If no one owns removing it, it tends to become permanent.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to scaling monolith horizontally, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.