What is the undici connection pool behind Node's fetch?

Node's global fetch is implemented by undici, an HTTP/1.1 client that ships inside the Node binary. Every fetch call is handed to a dispatcher. Unless you pass one explicitly, the request goes to a process-wide global dispatcher, which is an undici Agent. Undici is built from three layers:

  • A Client owns a single connection to a single origin (scheme, host and port).
  • A Pool manages a set of Clients for one origin. It opens connections up to a limit and assigns each request to a free one.
  • An Agent routes each request to a per-origin Pool, creating that Pool the first time the origin is seen.

The legacy http and https modules use none of this. Their http.Agent is a separate pool implementation, so fetch and http.request calls to the same host never share sockets. Settings applied to one stack have no effect on the other, and a socket leak, timeout or keep-alive bug in one stack does not show up in the other's metrics.

Aspect undici (fetch) http/https modules
Pool object Pool per origin, routed by Agent http.Agent
Default entry point Global dispatcher Global agent
Configuration Dispatcher options Agent options

Upstreams differ in latency, concurrency limits and idle timeouts, so tune a Pool or Agent dedicated to each origin. Changing the global dispatcher affects every outbound call in the process.

The bundled undici version depends on the Node release. Defaults and option names have changed between releases, so check the documentation for the undici version your Node binary actually includes.

undici stack (fetch)                        legacy stack (http/https)
  fetch(url)                                http.request()
      |                                           |
      v                                           v
+---------------------------+               +------------------------+
| Global dispatcher (Agent) |               | http.Agent             |
| routes by origin          |               | own socket pool        |
+---------------------------+               +------------------------+
      |               |                           |
      v               v                           |
+-------------+ +-------------+                   |
| Pool        | | Pool        |                   |
| origin A    | | origin B    |                   |
+-------------+ +-------------+                   |
    |        |           |                        |
    v        v           v                        v
+-------+ +-------+  +-------+              +------------------------+
|Client | |Client |  |Client |              | sockets, not shared    |
| conn  | | conn  |  | conn  |              | with fetch             |
+-------+ +-------+  +-------+              +------------------------+

The fetch path sends requests through the global dispatcher (an undici Agent) to per-origin Pools of single-connection Clients, while http.request uses http.Agent, which shares no sockets or settings with undici.

How does the pool work when a request is sent?

Following one request through the stack shows where each tuning option takes effect.

  1. Dispatch. fetch builds the request and passes it to the dispatcher, either the one given in the dispatcher option or the global one.
  2. Routing. An Agent looks up the Pool for the request's origin. If none exists yet, it creates one with the Agent's options. A Pool used directly skips this step because it only serves one origin.
  3. Connection selection. The Pool looks for a Client whose connection is open and idle. If it finds one, the request is assigned there and no handshake is needed. If every Client is busy and the Pool has fewer than connections Clients, it opens a new connection, paying TCP and TLS setup bounded by the connect timeout. If the limit has been reached, the request waits in the Pool's queue until a Client is free.
  4. Request and response. The Client writes the request and waits for response headers, bounded by headersTimeout. fetch resolves once headers arrive. The body streams afterwards, and bodyTimeout bounds the gap between chunks.
  5. Release. When the body has been fully read or cancelled, the Client is free again. The next queued request, if any, is assigned to it immediately. With pipelining: 1, a Client never carries more than one request at a time.
  6. Idle expiry. An idle connection starts a keep-alive timer. If the server sent a Keep-Alive: timeout=... hint, undici uses it, capped by keepAliveMaxTimeout. Otherwise it uses keepAliveTimeout. When the timer fires, the client closes the socket. The server or a load balancer may also close it first, which is the source of the stale-socket resets described below.
  7. Shutdown. close() stops accepting new requests, lets in-flight ones finish and then closes the sockets. destroy() aborts them immediately.

Most production problems map to one step. Leaked bodies block step 5, undersized pools build a queue at step 3, and keep-alive mismatches break step 6.

Why does keep-alive matter for outbound requests, and when does it fail?

A new connection to an HTTPS upstream costs at least one round trip for the TCP handshake and usually one or two more for TLS, before the request is sent. For a 5 ms internal call that overhead can exceed the call itself. Keep-alive lets the next request to the same origin reuse an open socket and skip both handshakes. The gain is largest for short, frequent calls to one origin, and smallest for rare or long-running requests.

Bodies must be consumed or cancelled

A socket goes back to the pool only after the response body has been fully read or explicitly cancelled. Code that checks res.status or res.ok and returns without touching the body leaves the connection busy until garbage collection eventually reclaims it. Under load, these leaked bodies look like pool exhaustion: requests queue, latency climbs, and the upstream reports nothing wrong. Every code path, including error branches, should either read the body (await res.json(), await res.text()) or discard it with await res.body?.cancel().

Stale sockets

The client and the server each close idle connections on their own timer. If the client keeps sockets longer than the server or load balancer does, it will sometimes reuse a socket the peer has already closed. The usual symptom is an intermittent ECONNRESET or other side closed error on the first request after a quiet period. To prevent this, set the client's keep-alive timeout below the shortest idle timeout on the path. Automatic retry on reset is safe only for idempotent requests. A POST may have reached the server before the socket failed, so retrying it can apply the operation twice.

import { Agent, fetch } from 'undici';

// Assumed: the load balancer in front of the upstream drops idle connections after 60s.
// Keep the client's idle timeout well below that so it never reuses a socket the peer closed.
const agent = new Agent({
  keepAliveTimeout: 10_000,    // close idle pooled sockets after 10s
  keepAliveMaxTimeout: 30_000, // cap any longer Keep-Alive hint sent by the server
});

async function getUser(id: string) {
  const res = await fetch(` { dispatcher: agent });
  if (!res.ok) {
    await res.body?.cancel(); // error path: discard the body so the socket is released
    throw new Error(`users service returned ${res.status}`);
  }
  return res.json(); // fully reading the body returns the socket to the pool
}

async function isHealthy() {
  const res = await fetch('', { dispatcher: agent });
  await res.body?.cancel(); // only the status matters, but the body must still be cancelled
  return res.status === 200;
}

// Retry once on a stale-socket reset, and only for idempotent methods.
// A POST may already have been processed before the socket failed.
const IDEMPOTENT = new Set(['GET', 'HEAD', 'PUT', 'DELETE', 'OPTIONS']);
const RESET_CODES = new Set(['ECONNRESET', 'UND_ERR_SOCKET']); // UND_ERR_SOCKET covers 'other side closed'

async function withResetRetry<T>(method: string, fn: () => Promise<T>): Promise<T> {
  try {
    return await fn();
  } catch (err: any) {
    const code = err?.cause?.code ?? err?.code; // fetch wraps network errors in a TypeError
    if (IDEMPOTENT.has(method) && RESET_CODES.has(code)) return fn();
    throw err;
  }
}

// Usage: const user = await withResetRetry('GET', () => getUser('42'));

An undici Agent whose keep-alive timeout is shorter than the upstream idle timeout, response bodies read or cancelled on every path, and a single retry on socket reset that applies only to idempotent methods.

Who should tune the pool, and when should you rely on the defaults instead?

Pool tuning pays off for a specific group: teams running services that make many outbound requests to a small set of internal upstreams, where connection setup cost is repeated thousands of times a minute. If that describes your service, the defaults are a starting point you should revisit, not a configuration you can forget about.

Four conditions justify the effort:

  • High-volume service-to-service traffic. At steady load, a pool that is too small queues requests inside your process. A pool that churns sockets pays TCP and TLS handshakes on every burst.
  • Latency-sensitive calls. When a request budget is tens of milliseconds, a single extra handshake round trip is a visible share of it.
  • Upstreams behind load balancers with known idle timeouts. If you know the timeout, you can set the client's keep-alive window below it instead of discovering it through intermittent resets.
  • Several upstreams with different needs. A shared global dispatcher applies one set of limits to a chatty internal API and a slow batch endpoint alike. Separate dispatchers let each get its own limits.

The global default is fine for low-volume scripts, CLIs, cron jobs and occasional third-party API calls. Connection reuse there is either already adequate or irrelevant to total runtime.

Do not tune the pool when the upstream itself is the bottleneck. If the server is saturated, more connections just move the queue from your process to its accept backlog or worker pool, and they often make tail latency worse. In that situation, deadlines, backpressure and circuit breaking protect you far more than connection counts do. Spend the engineering time there first.

When should pool tuning be avoided?

Beyond the saturated-upstream case, there are situations where custom pool settings add risk or maintenance without a matching benefit:

  • Short-lived or rarely active processes. A CLI or a job that makes a handful of requests and exits never reuses enough connections for the settings to matter. The extra configuration only adds code to maintain.
  • Runtimes that freeze the process between invocations. In environments that suspend a process while it is idle, pooled sockets can sit frozen past the upstream's idle timeout and fail on first reuse. Long keep-alive windows make this worse. Prefer short keep-alive timeouts and idempotent retries over larger pools there.
  • A proxy already owns pooling. If a sidecar or mesh proxy manages upstream connections, a large in-process pool to localhost adds nothing. Two layers of tuning also make it harder to tell which one causes a problem.
  • No way to observe the result. Without the pool gauges described below, you cannot tell whether a change helped. Blind changes to connection limits tend to hide leaked bodies or a slow upstream rather than fix them.
  • Shared libraries. A library that calls setGlobalDispatcher changes outbound behavior for every other library in the host application. Libraries should accept a dispatcher from the caller, not replace the global one.
  • Pipelining. Avoid enabling it for service calls for the reasons given in the next section.

In these cases, keep the defaults, consume or cancel every response body and put the effort into deadlines and error handling.

How many connections per origin do you need?

Little's law gives the starting point: average requests in flight equals arrival rate multiplied by average time in the system. With pipelining disabled, each in-flight request holds one connection, so that product is roughly the connection count a single process needs for one origin.

Illustrative estimate. Assume one pod sends 400 requests per second to an upstream with a 25 ms mean response time. That is 400 × 0.025 = 10 concurrent requests. The mean hides bursts and slow responses, so size against something closer to p99 latency, or add about 2× headroom. That gives a limit of about 20 connections.

Pool size Symptom Who pays
Too small Requests wait in the client queue, and latency rises before the upstream does any work Your service
Sized from Little's law plus headroom Steady reuse, little queuing Nobody
Unbounded Connection storms during slowdowns, file descriptor exhaustion, upstream overload Both sides

A per-pod limit is only half of the budget. Twenty connections across 30 replicas is 600 concurrent connections. If the upstream can serve 200 requests at a time, your fleet can overwhelm it while every individual pod looks reasonable. Divide the upstream's real capacity by your maximum replica count, including autoscaling peaks, and set the per-pod limit from that number.

Keep pipelining at 1 for service calls. With pipelining, one slow response blocks every request queued behind it on the same connection. If the connection drops, the client cannot tell which queued requests the server already processed. Retrying a POST in that state can duplicate a write. The gain is rarely worth these costs unless you control both the client and the server, so set pipelining: 1 explicitly rather than trusting a default.

How do you implement a dedicated dispatcher per upstream?

Create one Pool (single origin) or Agent (several origins) per upstream and pass it as the dispatcher option on each fetch call. Replacing the global dispatcher changes behavior for every library in the process. A dedicated dispatcher keeps limits and failures scoped to one dependency. Set connections from the sizing in the previous section. Set keepAliveTimeout below the server's or proxy's idle timeout. Otherwise the server can close a socket just as the client reuses it, and that request fails with a reset.

Timeouts work in separate layers:

Layer Enforced by Bounds
Connect timeout Dispatcher TCP and TLS setup
headersTimeout Dispatcher Wait for response headers
bodyTimeout Dispatcher Gap between body chunks
Request deadline Application (AbortSignal) Total time for the request

bodyTimeout resets on every chunk, so a slow trickle never trips it. Only an AbortSignal limits total latency.

Behind an L4 load balancer or a Kubernetes Service, the balancing decision happens once per connection. Long-lived sockets therefore stay pinned to whichever pods existed when they opened, and new replicas get little traffic. Capping requests or lifetime per connection (for example maxRequestsPerClient) forces reconnects that spread load again, at the cost of more handshakes.

Verify every option name against your installed undici or Node version. On shutdown, call close() on each dispatcher to drain in-flight requests. Keep construction, options and shutdown in one module. That way the extra configuration and lifecycle code is written and reviewed once.

// upstreams.js: the one place dispatchers are built, configured and closed.
// Option names match recent undici releases; check them against your installed version.
import { Pool, fetch } from 'undici';

const inventory = new Pool('', {
  connections: 32,             // from the sizing in the previous section
  pipelining: 1,               // one request per socket at a time; no pipelining
  keepAliveTimeout: 4_000,     // below the server's idle timeout (assumed 5s here)
  keepAliveMaxTimeout: 4_000,  // cap server keep-alive hints to the same bound
  connect: { timeout: 1_000 }, // TCP + TLS setup
  headersTimeout: 2_000,       // wait for response headers
  bodyTimeout: 2_000,          // max gap between body chunks, not total time
  maxRequestsPerClient: 1_000, // recycle sockets so L4 balancing happens again
});

const dispatchers = [inventory];

// The overall deadline is enforced here by the application, not by the pool.
export async function getItem(id, { deadlineMs = 3_000 } = {}) {
  const res = await fetch(` {
    dispatcher: inventory,
    signal: AbortSignal.timeout(deadlineMs),
  });
  if (!res.ok) throw new Error(`inventory responded ${res.status}`);
  return res.json();
}

// Call from the SIGTERM handler after the server stops accepting work.
// close() lets in-flight requests finish and rejects new ones.
export async function closeUpstreams() {
  await Promise.all(dispatchers.map((d) => d.close()));
}

This module creates a dedicated undici Pool for one upstream with explicit connection, keep-alive, pipelining and per-phase timeout settings, plus a connection recycle cap. Each call gets an AbortSignal deadline, and closeUpstreams() closes every pool during graceful shutdown.

How do you verify connection reuse and observe pool health?

Observability for the pool comes down to three signals: open connections per origin, queued versus running requests, and the ratio of reused connections to new ones. Recent undici releases expose a stats object on Pool and Agent instances, with counts for connected, free, pending, queued and running. Field names and availability have changed between releases, so check them against the documentation for the version in your lockfile. Export these counts as gauges labelled by upstream, which is straightforward if you already run one dispatcher per upstream.

Don't assume the client reuses connections. Confirm it from the other side. A packet capture filtered on the upstream port shows whether a burst of requests produces one TCP handshake or hundreds. Server-side access logs that record the client port, or a per-connection request count, show the same thing without a capture.

The signals separate the four common failures:

Symptom Running Queued New-connection rate Upstream view
Leaked bodies Stuck at the limit, stays high after traffic stops Grows Low Little or no in-flight work
Undersized pool Equal to the connection limit Grows under load, drains after Low Normal latency
Stale-socket resets Normal Normal Spikes after idle periods Closed idle sockets before the client did
Slow upstream High Grows Low High latency in its own logs

Stale-socket resets usually show up as ECONNRESET or "other side closed" errors on the first request after a quiet period. This points to a server keep-alive timeout that is shorter than the client's.

When an incident starts, these gauges reduce "outbound calls are slow" to one of four causes within minutes. Without them, engineers tend to raise connection limits, which makes a leaked-body problem worse and hides a slow upstream.

// One Pool per upstream so every gauge maps to a single origin.
import { Pool } from 'undici';

const upstream = ''; // placeholder origin
const pool = new Pool(upstream, { connections: 20 });

// Count new connections and disconnects. A burst of requests that keeps
// adding to newConnections means sockets are not being reused.
let newConnections = 0;
let disconnects = 0;
pool.on('connect', () => { newConnections++; });
pool.on('disconnect', (origin, targets, err) => {
  disconnects++;
  // ECONNRESET / 'other side closed' right after idle points at keep-alive mismatch
  if (err) console.warn('disconnect', String(origin), err.code ?? err.message);
});

// Sample pool stats periodically. Field names vary by release, so check
// them against the docs for the undici version in your lockfile.
setInterval(() => {
  const s = pool.stats;
  if (!s) return; // older releases do not expose stats
  console.log(JSON.stringify({
    upstream,
    connected: s.connected, // open sockets
    free: s.free,           // idle sockets ready for reuse
    running: s.running,     // requests in flight
    pending: s.pending,     // waiting for a socket (queued + not yet dispatched)
    queued: s.queued,
    newConnections,         // should stay flat under steady load
    disconnects,
  }));
  // In production, export these as gauges/counters labelled by upstream
  // instead of logging them.
}, 10_000).unref();

export { pool };

Samples a per-upstream undici Pool's stats and counts connect/disconnect events so running, queued and new-connection rates can be exported and compared against the failure table.

How can pool tuning save engineering time?

Most of the time saved comes from avoiding investigations, not from faster requests.

  • Fewer intermittent-error investigations. Stale-socket resets appear rarely, cluster after quiet periods and seldom reproduce locally, so they can take a long time to track down. Setting the keep-alive timeout below the upstream's idle timeout and retrying idempotent requests once removes most of them before anyone has to investigate.
  • Faster incident triage. With per-upstream gauges and the failure table above, the on-call engineer can tell a leaked body, an undersized pool, a keep-alive mismatch or a slow upstream apart from metrics alone. Without them, triage usually starts with packet captures and guesswork.
  • Fewer counterproductive fixes. Teams that cannot see the pool tend to raise connection limits, redeploy and repeat. Sizing from Little's law and a fleet-wide budget gives a number to defend in review instead of trial and error under pressure.
  • One place to change. Keeping dispatcher construction, options and shutdown in a single module means a new upstream timeout or limit is a one-line change, reviewed once, instead of a search through every call site that passes ad-hoc options.
  • Contained failures. A dedicated dispatcher per upstream keeps one slow dependency from exhausting connections that other calls need. That narrows the blast radius and shortens the list of suspects during an outage.
  • Cleaner deploys. Calling close() during graceful shutdown lets in-flight requests finish instead of failing mid-deploy, which removes a recurring source of noisy errors after each release.

The upfront cost is small: one module per service, a few gauges and a check of the upstream's idle timeout. It is worth paying only for services that meet the conditions listed earlier.

What are the alternatives to tuning undici directly, and what do they cost?

Tuning dispatchers in-process is one option among four, and each one shifts cost to a different place.

Option Per-upstream control Ongoing cost Retry amplification risk Load balancing Coupling to Node release
Global default dispatcher None; one config for all origins Lowest Low; no retries unless you add them Per connection, follows DNS High; fetch's bundled undici ships with Node
http.Agent with http/https Per agent, coarse Medium; second client stack to maintain Depends on your wrapper Per connection Low; long-stable API
undici request API Full, per dispatcher Medium; a dependency you upgrade yourself Explicit if you use retry handlers Per connection Low if pinned
Sidecar or mesh proxy Full, in proxy config Shifts to the platform team High if app and proxy both retry Per request, L7 None

Global default dispatcher. This is the cheapest option. Every upstream shares one set of limits, so one slow dependency can push the pool settings in a direction that hurts the others.

http.Agent with the http/https modules. Legacy services already use it, but it gives you a second client stack and a second set of keep-alive semantics to reason about.

undici's own request API. It removes the overhead of fetch's spec compliance and gives you the most control. The cost is that you own the dependency and its upgrades.

Sidecar or mesh proxy. Here the app keeps a few connections to localhost, and the proxy balances each request across endpoints. That fixes the problem of long-lived connections pinning traffic to a few backends. The cost is an extra hop, plus retry policy that lives outside the code.

The main risk with a proxy is stacked retries. The worst-case number of attempts against the upstream is app_attempts × proxy_attempts.

Illustrative estimate (assumed numbers, not a benchmark): with 3 app attempts and 3 proxy attempts, each failing call becomes 3 × 3 = 9 upstream attempts.

Choosing an option:

  • Low volume, few upstreams: keep the global dispatcher.
  • Few hot upstreams with different latency profiles: use a dedicated dispatcher per upstream.
  • Many replicas behind DNS and an existing mesh: let the proxy handle pooling, and disable app-level retries.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to undici connection pool, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.