Nginx Upstream Keepalive and Timeouts for Node.js Backends

What it is, who should use it, and how it works

What is it?

Upstream keepalive is Nginx's connection pooling for backend servers. When you add the keepalive directive to an upstream block, each Nginx worker process caches idle HTTP/1.1 connections to the backends and reuses them for later proxied requests. Without it, Nginx opens a new connection for each request. Turning it on safely also means aligning three sets of timeouts: Nginx's client-facing idle timeout, Nginx's upstream idle timeout, and the Node.js http.Server timeouts. This article covers that configuration and ordering, how to diagnose the 502s that appear when the ordering is wrong, and how to verify the fix.

Who should use it?

• Teams running a Node.js HTTP server behind Nginx as a reverse proxy, especially at moderate or high request rates. At those rates, per-request TCP handshakes and TIME_WAIT buildup start to cost CPU, latency and ephemeral ports.
• Services with bursty traffic. Quiet periods followed by spikes are the conditions that expose the idle-timeout race.
• Anyone seeing intermittent 502s with "upstream prematurely closed connection" or "Connection reset by peer" in the Nginx error log, particularly on the first requests after a lull.

The benefit is smaller for low-traffic services, where a handshake per request costs little. It also needs extra care in multi-hop chains, where every hop's timeouts must be ordered. The section on when to avoid or limit these changes covers those cases.

How does it work?

1. A client request arrives at an Nginx worker. The worker picks a backend server from the upstream block using the configured balancing method.
2. The worker checks its own cache for an idle connection to that server. If it finds one, it sends the request on that socket. If not, it opens a new TCP connection.
3. The request goes out as HTTP/1.1 with the Connection header cleared, so neither side asks to close the connection after the response.
4. When the response finishes and the connection is still reusable, the worker returns it to its idle cache. If the cache already holds keepalive idle connections, the least recently used one is closed.
5. An idle cached connection is closed when the upstream keepalive_timeout expires. It is also closed when it has served the number of requests set by the upstream keepalive_requests directive, or when the backend closes it.
6. On the Node side, http.Server tracks idle sockets and closes any socket that has had no new request within server.keepAliveTimeout.

The weak point is step 2. Nginx removes a cached connection when it sees the backend's close. If Node's close is still in flight when Nginx picks that socket, the request is written onto a connection that is already going away. The fix is to order the timeouts so that Nginx always retires an idle socket before Node's timer can fire. The rest of this article explains how.

What upstream keepalive is and why idle timeouts cause 502s

By default, Nginx proxies each request over a fresh HTTP/1.0 connection to the upstream and sends Connection: close, so the TCP connection is torn down after every response. Under load, that means a TCP handshake per request, sockets piling up in TIME_WAIT, and possible ephemeral port exhaustion on the proxy host. The keepalive directive in an upstream block changes this. Each worker process keeps a pool of idle connections to the backend and reuses them for later requests. This only works with proxy_http_version 1.1 and a cleared Connection header.

Reuse introduces a race. Nginx keeps an idle socket in its pool and assumes it is still usable. If Node's idle timer fires first, Node closes the socket. Nginx may then write a request onto that socket just as the close arrives. The request fails, Nginx logs something like "upstream prematurely closed connection", and the client gets a 502.

The three timeout layers

• Client keepalive_timeout (in an http or server block) controls how long Nginx holds idle connections from browsers and load balancers. It does not affect upstream sockets.
• Upstream idle timing (keepalive_timeout inside the upstream block) controls how long Nginx keeps a pooled backend connection open before closing it.
• Node's server.keepAliveTimeout is how long Node waits on an idle socket before closing it. headersTimeout limits how long Node waits for the request headers to arrive. requestTimeout limits how long it waits for the whole request. Node's idle default is only a few seconds, which is shorter than Nginx's.

The core rule is that Node's keepAliveTimeout must be longer than Nginx's upstream idle timeout. That way Nginx always closes an idle socket before Node does, and Nginx never sends a request onto a connection Node has already closed.

Teams running Node behind Nginx with steady or bursty traffic benefit most. Bursty traffic is especially exposed: a quiet gap lets many pooled sockets age past Node's timeout, and the next burst hits them all at once.

Copy +-----------+      +---------------------------+      +------------------+
 |  Client   |----->|  Nginx worker process     |----->|  Node.js server  |
 | (browser/ |<-----|  proxy_http_version 1.1   |<-----|  (http.Server)   |
 |    LB)    |      |  proxy_set_header         |      |                  |
 +-----------+      |    Connection ""          |      +------------------+
                    |  +---------------------+  |
                    |  | upstream idle pool  |  |
                    |  | [conn] [conn] [conn]|  |
                    |  +---------------------+  |
                    +---------------------------+

 [1] Client side    [2] Upstream idle                  [3] Node timeouts
 keepalive_timeout  keepalive_timeout                  server.keepAliveTimeout
 (http/server blk)  (inside upstream {})               headersTimeout
                                                       requestTimeout

 Race if [3] Node idle timeout <= [2] Nginx upstream idle timeout:

 Nginx --- socket idle in pool -------- reuse --> send request --X 502
 Node  --- socket idle --- timer fires --> close socket --------^

 Rule: [3] Node keepAliveTimeout  >  [2] Nginx upstream keepalive_timeout
       => Nginx closes idle sockets first; no request lands on a closed one.

Nginx keeps a per-worker pool of idle HTTP/1.1 connections to Node. When Node's keepAliveTimeout is not longer than Nginx's upstream idle timeout, Node can close a pooled socket just as Nginx reuses it, and the client gets a 502.

Configuring Nginx and Node so Nginx always retires idle sockets first

Start in the upstream block. The keepalive directive turns on connection caching to the backend. Nginx only reuses those sockets if the proxied requests are HTTP/1.1 without a Connection: close header. Set proxy_http_version 1.1 and proxy_set_header Connection "" in the location that proxies to Node. If you skip either one, Nginx accepts the keepalive setting but still opens a new connection per request.

Next, set keepalive_timeout inside the upstream block. This is separate from the client-facing keepalive_timeout in the http or server context. The upstream version controls how long Nginx holds an idle backend socket. Keep it shorter than Node's server.keepAliveTimeout, with a margin of a few seconds. The side that times out first closes the socket. You want that to be Nginx, because Nginx won't send a request on a socket it has already retired.

On the Node side:

• Raise keepAliveTimeout above the Nginx value.
• Set headersTimeout above keepAliveTimeout. This stops a reused connection from hitting the header timer while Node still considers it alive.

keepalive is not a connection limit. It is the maximum number of idle connections each worker process caches for that upstream. Nginx opens more connections under load and closes the extras when they go idle. The total idle pool can reach worker_processes × keepalive, spread across your Node instances. For example, eight workers with keepalive 32 can hold 256 idle sockets. Check that your Node processes and file descriptor limits can handle that number.

Defaults have changed between releases. Check the documentation for your installed Nginx and Node versions before you rely on any value, including the ones in this article.

Copyupstream node_app {
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;

    # Per-worker cap on IDLE cached connections, not a total limit.
    # Nginx opens more under load and closes the extras once idle.
    keepalive 32;

    # How long Nginx keeps an idle backend socket.
    # Keep this a few seconds BELOW Node's server.keepAliveTimeout
    # so Nginx always retires the socket first. Example Node values:
    #   server.keepAliveTimeout = 65000;  // ms
    #   server.headersTimeout   = 66000;  // ms, above keepAliveTimeout
    keepalive_timeout 60s;
}

server {
    listen 80;
    server_name app.example.internal;

    location / {
        proxy_pass

        # Both are required for upstream reuse. Without them Nginx
        # accepts 'keepalive' but opens a new connection per request.
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_set_header Host $host;
    }
}

Nginx upstream keepalive config with an upstream keepalive_timeout set below the Node keepAliveTimeout and headersTimeout values shown in the comments, so Nginx closes idle sockets before Node does.

Which proxy timeout controls which failure

Nginx has three upstream timers, and each one guards a different phase of the proxied exchange. None of them caps the total duration of a request. proxy_read_timeout and proxy_send_timeout measure the gap between two successive read or write operations. A response that streams one byte every few seconds can therefore run indefinitely without tripping proxy_read_timeout.

Note that keepalive_timeout in the http or server context faces the client. It has no effect on Nginx-to-Node sockets. The idle lifetime of upstream connections is set by the separate keepalive_timeout inside the upstream block, which the previous section aligned with Node's own timeout.

Raising proxy_read_timeout globally to accommodate one slow endpoint hides stalled requests everywhere else and keeps worker connections occupied. A narrower approach is to override it per location:

• Long-polling endpoints need a read timeout slightly longer than the server's maximum hold time.
• SSE endpoints need a read timeout longer than the heartbeat interval, plus proxy_buffering off so events are not held back.
• Report endpoints need a generous read timeout. A better fix is to move that work to a background job.

Copyupstream node_app {
    server 127.0.0.1:3000;
    keepalive 32;
    keepalive_timeout 55s;           # upstream idle sockets, shorter than Node's keepAliveTimeout
}

server {
    listen 80;
    keepalive_timeout 65s;           # client-facing idle timeout only

    proxy_http_version 1.1;
    proxy_set_header Connection "";  # allow upstream connection reuse
    proxy_connect_timeout 5s;        # fail fast if Node can't accept
    proxy_send_timeout 30s;          # max gap between writes to Node
    proxy_read_timeout 30s;          # max gap between reads; default for normal API calls

    location / {
        proxy_pass
    }

    location /poll/ {
        proxy_pass
        proxy_read_timeout 35s;      # server holds polls up to 30s; allow a small margin
    }

    location /events/ {
        proxy_pass
        proxy_buffering off;         # deliver SSE frames immediately
        proxy_read_timeout 60s;      # comfortably above the 15s heartbeat interval
    }

    location /reports/ {
        proxy_pass
        proxy_read_timeout 300s;     # handler sends nothing until the report is built
    }
}

Conservative server-wide upstream timeouts with per-location proxy_read_timeout overrides for long-polling, SSE and slow report endpoints, alongside the separate client-facing and upstream keepalive_timeout settings.

Diagnosing 502s from the Nginx error log

Every 502 or 504 that Nginx returns leaves a line in the error log, and the wording of that line tells you which side gave up. Search for the message text rather than the status code, because the same 502 can come from very different causes.

The message alone does not settle the cause. You also need to know what was happening around it. Line up the error timestamps against these events:

• Process restarts. Check your supervisor or orchestrator logs for restarts at those timestamps. A cluster of "connection refused" errors that lines up with a restart points to a crash, not a misconfiguration.
• Deploys. Resets and premature closes that appear only during a rollout usually mean old workers are exiting without draining in-flight requests.
• Event-loop stalls. Timeouts that coincide with CPU spikes or garbage-collection pauses suggest a blocked loop, not a slow downstream dependency.

If none of these events line up, look at traffic timing. Premature closes that land on the first request after a quiet period are the signature of the keepalive race. Those errors appear regardless of load, which rules out capacity as the cause.

Copy#!/usr/bin/env bash
set -euo pipefail

LOG="${1:-/var/log/nginx/error.log}"   # Nginx error log to scan
UNIT="${2:-node-api.service}"          # systemd unit running Node
SINCE="${3:-2 hours ago}"               # window for restart events

echo "== Upstream errors per minute (count, date, HH:MM, class) =="
awk '
  { c = "" }
  /\(111: Connection refused\)/   { c = "refused" }          # Node not listening
  /upstream timed out/            { c = "timeout" }          # slow handler / blocked loop
  /upstream prematurely closed/   { c = "premature_close" }  # keepalive race or exit
  /\(104: Connection reset by peer\)/ { c = "reset" }        # same, but reset
  c != "" { print $1, substr($2, 1, 5), c }
' "$LOG" | sort | uniq -c

echo
echo "== Start/stop events for $UNIT since '$SINCE' =="
journalctl -u "$UNIT" --since "$SINCE" -o short-iso --no-pager \
  | grep -Ei 'start|stop|exit|kill' || echo "(no start/stop events found)"

A shell script that counts Nginx upstream errors by minute and by message type, then prints the Node service's systemd start and stop events for the same window so you can match error bursts to restarts and deploys.

Verifying the fix: connection reuse and idle-then-burst testing

Confirming connections are reused

Under steady traffic, the set of established connections from Nginx to the Node port should stay roughly constant. On the Nginx host, run ss -tn state established '( dport = :3000 )' (substitute your Node port) a few times, several seconds apart. With reuse working, the same local ephemeral ports appear in each sample, and the count stays near your keepalive pool size per worker. If the source ports change on every sample, Nginx is opening a new connection per request. That usually means proxy_http_version 1.1 or the cleared Connection header is missing. A rising count from ss -tan state time-wait '( dport = :3000 )' points to the same problem. On hosts without ss, netstat -tn filtered on the port shows the same information.

Idle-then-burst testing

Steady load cannot reproduce this bug, because pooled sockets never sit idle long enough for Node to close them. The test needs a pause instead:

1. Send a short warm-up burst to fill the pool.
2. Stay silent for longer than Node's keepAliveTimeout. If it is 5 seconds, wait 8 to 10 seconds.
3. Burst again.
4. Repeat the cycle dozens of times. The race depends on timing, so a single clean pass proves little.

Use POST requests for the burst. Nginx does not retry non-idempotent requests on the next upstream by default, so the race cannot be hidden by a successful retry.

Before the fix, expect occasional 502s on the first requests after each pause, along with matching "upstream prematurely closed" entries in the error log. After the fix, both should be absent across every cycle.

When to avoid or limit these changes, and the alternatives

Keepalive pools are not free. Each idle upstream connection holds a file descriptor and some memory on the Node process. If the keepalive count is multiplied across many Nginx workers and instances, a small backend can end up holding thousands of mostly idle sockets. Size the pool to steady-state concurrency, not peak.

A long keepAliveTimeout in Node also affects every client that connects directly, not only Nginx. Health checkers and load balancers that bypass Nginx will keep their sockets open just as long.

The ordering rule must hold at every hop. In a chain of cloud load balancer → Nginx → sidecar proxy → Node, each caller's idle timeout must be shorter than the next hop's. If one link is misordered, the 502s come back.

Deploys are a separate problem. A cluster or PM2 restart that kills workers abruptly drops in-flight and pooled connections, whatever the timeouts are set to.

Alternatives

Graceful shutdown complements the timeout ordering and does not replace it. Most production setups need both.

Impact on cost and engineering time

The larger savings are usually in engineering time, not compute. A timeout race produces 502s that appear only after idle periods, clear up under steady load, and rarely show up in staging. Teams can lose days to these before anyone checks the timeout ordering. When the ordering is correct and written down, that whole category of incident is ruled out.

Illustrative estimate (assumed numbers, not a benchmark)

Assume 2,000 upstream requests/s, a 95% connection reuse ratio once keepalive works, a 60 s TIME_WAIT period, and 0.5 ms network RTT.

• Handshakes avoided per second = request rate × reuse ratio = 2,000 × 0.95 = 1,900
• Sockets in TIME_WAIT = new connections/s × TIME_WAIT seconds
  • Without reuse: 2,000 × 60 = 120,000, which is more than a typical ephemeral port range of tens of thousands for one destination.
  • With reuse: 100 × 60 = 6,000.
• Latency removed per reused request ≈ 1 RTT ≈ 0.5 ms. If TLS runs to the upstream, it removes more.

The cost of masking the problem

Raising every timeout or adding blanket retries hides the symptom. Fast 502s become slow 504s. Retries on non-idempotent requests can turn one failed POST into a duplicate charge or a duplicate order. Those incidents cost more to clean up than the original 502s did.

Checklist for every new hop (load balancer, ingress, sidecar, proxy)

• Record each hop's idle timeout in one shared document.
• Confirm that each client-side idle timeout is shorter than the server-side timeout of the hop behind it.
• Allow retries only for idempotent methods.
• Run the idle-then-burst test before release.
• Require reviewers to check the timeout doc when proxy config changes.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nginx upstream keepalive, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.