Undici Connection Pooling for Node.js Fetch: Sizing and Timeouts

How Node's fetch uses undici's dispatcher, Client, Pool and Agent

Node's global fetch is implemented on top of undici, an HTTP/1.1 client written for Node. Your code never opens sockets directly. Each fetch call goes to a dispatcher, the undici abstraction that accepts a request and decides which connection carries it. Unless you change it, that dispatcher is a process-wide default: an Agent that keeps a separate keep-alive pool for each origin (scheme, host and port) you call.

The object model

• Client manages connections to a single origin and queues requests waiting for them.
• Pool groups several Clients for one origin and spreads requests across them. Its connections option sets how many sockets it may open.
• Agent maps origins to Pools and creates a Pool the first time a request targets a new origin.

Why the connection limit caps concurrency

An HTTP/1.1 connection carries one request at a time. Undici supports pipelining, which sends several requests on one socket, but it is off by default. Most servers and proxies handle it poorly, so you should usually leave it off.

With pipelining off, the per-origin connection limit is your real concurrency limit for that origin. Request number N+1 waits in the queue until a socket frees up. That queue time is counted in your latency, but the server never sees it.

Replacing the dispatcher

You can swap the default in two ways:

• Call setGlobalDispatcher with your own configured Agent to change the default for every fetch in the process.
• Pass a dispatcher option to a single fetch call to route only that request.

Node bundles its own copy of undici. If you install undici from npm, its version may differ from the bundled one. Defaults such as connection counts and timeouts, along with option names, have changed across Node and undici releases. Check them against the exact versions you run.

Copy// npm install undici  (check that its version is compatible with your Node release)
import { Agent, Pool, setGlobalDispatcher } from 'undici';

// Replace the process-wide default dispatcher used by global fetch.
// Agent keeps one Pool per origin; these options apply to each of those Pools.
const agent = new Agent({
  connections: 20,          // max sockets per origin = max concurrent requests per origin
  pipelining: 1,            // one request per socket at a time (no HTTP/1.1 pipelining)
  keepAliveTimeout: 10_000  // ms an idle socket is kept open for reuse
});
setGlobalDispatcher(agent);

// Every fetch now goes through `agent`. Requests beyond 20 in flight to
// api.example.com wait in undici's queue, not on the server.
const res = await fetch('');
console.log(res.status);

// Route a single request through a dedicated Pool for one origin,
// e.g. a slow reporting service that should not use up the shared sockets.
const reportsPool = new Pool('', {
  connections: 4
});

const report = await fetch('', {
  dispatcher: reportsPool   // overrides the global dispatcher for this call only
});
console.log(report.status);

// Close pools on shutdown so idle keep-alive sockets don't hold the process open.
await reportsPool.close();
await agent.close();

Sets a global undici Agent with an explicit per-origin connection limit for Node's fetch, and routes one request through a dedicated Pool using the per-request dispatcher option.

Why pooling needs tuning, and who should tune it

Every new connection to an HTTPS upstream costs a TCP handshake plus a TLS handshake before the first request byte is sent, which adds several network round trips, and the TLS step also costs CPU on both ends. Reusing an open socket skips all of that. Pooling also limits socket churn. Each closed connection leaves the client holding a local port in TIME_WAIT for a while, so a service that opens and closes connections at high rates can exhaust its ephemeral ports or hit file-descriptor limits.

This matters most to backend and platform engineers whose services make sustained outbound HTTP calls to a small number of upstreams, such as an internal API, a payment provider or an object store. With traffic concentrated on a few origins, the per-origin pool decides latency and failure behaviour.

Treating fetch as a stateless one-liner hides that pool, and the symptoms rarely point back to it:

• Socket churn. Connection counts and handshake CPU climb with traffic instead of levelling off.
• Stalled requests. Requests wait for a free connection, often because earlier response bodies were never consumed. Latency spikes without any matching slowness upstream.
• Sporadic ECONNRESET. The client reuses a socket the server has just closed for being idle. These errors look like network flakiness and often get papered over with blind retries.

Keep-alive is already on by default, so there is nothing to switch on. The real work is to size the pool, scope it to the right upstreams, and set timeouts that match the servers you call.

Keep-alive, idle timeouts and the stale-socket race

…what the upstream actually does with idle connections.

Undici's keepAliveTimeout controls how long an idle socket stays in the pool before the client closes it. The server runs its own idle timer. If the server's timer fires first, it closes the socket, and its FIN can be in flight at the moment the client writes a new request onto that connection. The client then sees a reset or "other side closed" error on a request that never reached application code. Undici honors a Keep-Alive: timeout= response header and subtracts keepAliveTimeoutThreshold to leave a safety margin, but many servers do not send that header. Proxies, load balancers and NAT gateways never send it.

The rule: the client's idle timeout must be shorter than the shortest idle timeout on the path. That includes the upstream server and every proxy, load balancer and NAT device in between. Look up each of those values and stay below the lowest one. Do not copy a number from a blog post.

Middleboxes are the dangerous case because they often drop idle flows without sending any packet. The socket still looks healthy to Node, so the next request waits for a response that will never come and fails only when your request timeout fires. Long idle timeouts avoid repeated TLS handshakes, but every extra idle second raises the chance of reusing a socket that is already dead.

Long-lived connections also cause a routing problem. A layer-4 load balancer picks a backend once per connection, so a warm pool can keep sending most of its traffic to the same few backends while new instances get little. The pool also holds connections to IP addresses resolved before a DNS change. To counter both, recycle connections periodically: on a schedule, replace the dispatcher and close the old one gracefully, so new connections get re-resolved and re-balanced.

Copyimport { Agent, fetch } from 'undici';

// Placeholder: the LOWEST idle timeout on the path (upstream server, proxy,
// load balancer, NAT). Look these up for your environment.
const SHORTEST_PATH_IDLE_MS = 60_000;
const MARGIN_MS = 5_000; // stay safely below it
const RECYCLE_EVERY_MS = 5 * 60_000; // rebalance L4 backends, pick up DNS changes

function makeAgent(): Agent {
  return new Agent({
    connections: 50,
    // Close idle sockets before anything on the path does.
    keepAliveTimeout: SHORTEST_PATH_IDLE_MS - MARGIN_MS,
    // Cap any longer value the server advertises via Keep-Alive: timeout=.
    keepAliveMaxTimeout: SHORTEST_PATH_IDLE_MS - MARGIN_MS,
    // Subtracted from a server-advertised timeout as a safety margin.
    keepAliveTimeoutThreshold: 2_000,
  });
}

let dispatcher = makeAgent();

// Periodic recycling: new requests get fresh connections (new DNS lookup,
// new backend choice); the old agent drains in-flight requests, then closes.
const timer = setInterval(() => {
  const old = dispatcher;
  dispatcher = makeAgent();
  old.close().catch((err) => console.error('agent close failed', err));
}, RECYCLE_EVERY_MS);
timer.unref(); // don't keep the process alive just for recycling

export async function getJson(url: string): Promise<unknown> {
  // Request timeout bounds the wait on a socket a middlebox silently dropped.
  const res = await fetch(url, { dispatcher, signal: AbortSignal.timeout(10_000) });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  return res.json();
}

An undici Agent whose idle timeout is set below the shortest idle timeout on the network path. It is replaced on a schedule and the old agent is closed gracefully, so traffic rebalances across L4 backends and picks up DNS changes.

Sizing the pool with Little's law and fleet-wide limits

Little's law gives the starting point: concurrent connections ≈ request rate × mean latency. Illustrative estimate: with an assumed 200 req/s to one upstream at an assumed mean latency of 50 ms, about 10 connections are busy at any moment.

That figure is only a floor. Requests arrive in bursts, and the upstream's tail sets the real demand. If p99 latency spikes to an assumed 400 ms during a slow deploy, the same 200 req/s needs about 80 connections. With a cap of 10, requests queue inside the dispatcher, and that queueing time adds to every caller's latency. Size the pool for the latency you see under stress.

Then multiply. At an assumed 50 connections per pod across 30 pods, the fleet can open 1,500 sockets. Compare that number against the upstream's connection limit, your load balancer's per-target cap and any egress proxy's limit. Autoscaling raises the total without anyone editing the config.

Keep pipelining at 1, which is undici's default. One slow response stalls every request queued behind it on the same socket, and many servers and proxies handle pipelining poorly. If you need more concurrency, add connections.

Implementing it correctly: body consumption, timeouts and retries

Each response holds its connection until the body is fully read or cancelled. fetch resolves as soon as headers arrive. Code that checks res.ok, throws, and walks away leaves the socket checked out until garbage collection or bodyTimeout reclaims it. Under load, this looks like a pool that gradually stops sending requests.

The rule is that every path must either consume the body (await res.json(), await res.arrayBuffer()) or call await res.body?.cancel(). That includes non-2xx branches and catch blocks. The simplest way to enforce it is a finally block that cancels any unread body.

Socket-level timeouts measure gaps, not totals. A server that sends a byte just inside bodyTimeout never trips it. Time spent queued waiting for a free connection counts against none of them. Only a per-call signal enforces the deadline your caller actually has.

A stale keep-alive socket typically fails with a reset or "other side closed" error. Retrying GET, PUT and DELETE is usually safe. For a POST, you can't know whether the server processed it, so retry only when the request carries an idempotency key.

Put retries in one wrapper around the client, not at each call site, and don't stack them on top of a proxy's retries. Share a single deadline across all attempts.

Each retry is another checkout from the pool:

connections needed ≈ request rate × latency × (1 + retry fraction)

Illustrative estimate (assumed numbers, not a benchmark): 200 req/s × 0.1 s × 1.2 = 24 connections, not 20.

Copyimport { Agent, fetch } from 'undici';

// One shared pool per process. Socket-level timeouts bound gaps, not totals.
const dispatcher = new Agent({
  connections: 50,               // max sockets per origin
  connect: { timeout: 5_000 },   // TCP + TLS handshake
  headersTimeout: 10_000,        // request sent -> headers received
  bodyTimeout: 15_000,           // idle gap between body chunks
});

const STALE = new Set(['ECONNRESET', 'UND_ERR_SOCKET']); // reset / other side closed
const IDEMPOTENT = new Set(['GET', 'HEAD', 'PUT', 'DELETE', 'OPTIONS']);

// Single wrapper owns retries. Headers are assumed to use lowercase keys.
export async function requestJson(url, { method = 'GET', headers = {}, body, deadlineMs = 8_000, maxRetries = 1 } = {}) {
  // One wall-clock deadline shared by every attempt, including time queued for a socket.
  const signal = AbortSignal.timeout(deadlineMs);
  // POST is retried only when it carries an idempotency key.
  const retryable = IDEMPOTENT.has(method) || 'idempotency-key' in headers;

  for (let attempt = 0; ; attempt++) {
    let res;
    try {
      res = await fetch(url, { method, headers, body, signal, dispatcher });
      if (!res.ok) throw new Error(`HTTP ${res.status} from ${url}`);
      return await res.json(); // reading the body releases the connection
    } catch (err) {
      const code = err.cause?.code ?? err.code;
      // Retry only connection-level failures that happened before any response arrived.
      if (!res && retryable && STALE.has(code) && attempt < maxRetries && !signal.aborted) continue;
      throw err;
    } finally {
      // Every path, including non-2xx and errors, gives the socket back to the pool.
      if (res && !res.bodyUsed) await res.body?.cancel().catch(() => {});
    }
  }
}

A single undici-backed wrapper that sets socket-level timeouts on a shared Agent, applies one AbortSignal deadline across all attempts, retries stale-socket errors only for idempotent requests, and always consumes or cancels the response body.

When to use one global Agent, dedicated dispatchers, or neither

A single global Agent is enough when your upstreams look alike: similar latency, the same TLS trust settings, and timeouts that suit all of them. An Agent already keeps a separate pool per origin, so two hosts do not compete for the same sockets. For many services, setting one tuned Agent with setGlobalDispatcher at startup is the whole job.

Dedicated dispatchers earn their cost when settings have to differ. An Agent applies one headersTimeout, one keepAliveTimeout and one per-origin connections cap to every origin. If one dependency needs a 30-second timeout, a single Agent forces that timeout onto everything else. Different TLS requirements also call for a separate dispatcher, such as a client certificate or a private CA for one internal service. The subtler case is a single origin serving both a fast lookup endpoint and a slow export endpoint. They share one pool, so slow exports can hold every connection while fast lookups wait in the queue. Give the slow traffic its own Pool and pass it through fetch's dispatcher option. That caps how much of the connection budget it can take.

Create every dispatcher once, at module or application scope. On shutdown, call close() to let in-flight requests drain, or call destroy() to abort them. A dispatcher built per request never reuses a socket, so you lose keep-alive. It can also leak open handles that keep the process alive.

Skip tuning for low-volume or one-off calls, short scripts, and serverless functions where sockets rarely survive between invocations. The defaults are fine there.

Each extra dispatcher adds more settings to keep in sync, more sockets to account for against fleet-wide limits, and one more object to close. Add one only when you can name the dependency it isolates.

Copyimport { readFileSync } from 'node:fs';
import { Agent, Pool, fetch, setGlobalDispatcher } from 'undici';

// Created once at module scope. This is the default for every origin.
const defaultAgent = new Agent({
  connections: 50,          // per-origin socket cap
  keepAliveTimeout: 10_000,
  headersTimeout: 5_000,
});
setGlobalDispatcher(defaultAgent);

// Slow export endpoint on the same origin gets its own small pool,
// so long exports cannot hold the sockets that fast lookups need.
const exportPool = new Pool('', {
  connections: 4,
  headersTimeout: 30_000,
  bodyTimeout: 120_000,
});

// Internal service signed by a private CA: separate TLS settings.
const billingAgent = new Agent({
  connect: { ca: readFileSync('/etc/ssl/private-ca.pem') },
});

export const getUser = (id) =>
  fetch(`); // global Agent

export const runExport = () =>
  fetch('', { dispatcher: exportPool });

export const getInvoice = (id) =>
  fetch(` { dispatcher: billingAgent });

// On shutdown, close() lets in-flight requests drain. Use destroy() to abort them.
process.once('SIGTERM', async () => {
  await Promise.all([defaultAgent.close(), exportPool.close(), billingAgent.close()]);
});

This example creates one global Agent, a dedicated Pool for a slow endpoint and an Agent with a private CA, all at module scope, then closes all three on shutdown.

Observability and alternatives: diagnostics_channel, the undici API and other clients

Undici publishes lifecycle events on node:diagnostics_channel. These include undici:request:create, undici:request:headers, undici:request:error, undici:client:beforeConnect, undici:client:connected and undici:client:connectError. Channel names and payloads have changed between releases, so check them against the undici version bundled with your Node runtime before you build dashboards on them.

None of these events reports reuse directly. You can derive it: compare the count of client:connected with the count of request:create per origin. A rising connect-to-request ratio means sockets are being closed and reopened instead of reused.

To tell a sizing problem from an upstream problem, split the latency:

• Sizing problem: time from request:create to headers sent grows while server time stays flat. This means requests are waiting for a free connection.
• Upstream problem: server time grows, or you see connectError and ECONNRESET errors while queue time stays near zero. Adding connections will not fix this.

Use fetch when code is shared across runtimes. Use undici's own APIs for hot paths where per-call control matters. Use http.Agent mainly for legacy clients built on node:http.

The payoff is engineering time. With measured pools, failures stop looking like random flakiness. Intermittent ECONNRESET errors and hung requests become specific causes you can diagnose, so you spend fewer hours chasing them.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to undici connection pooling, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.