Node.js Worker Threads for CPU-Bound Work in API Servers

How to confirm event loop blocking before you change anything

Worker threads add complexity, so first prove the problem is synchronous computation and not slow I/O. Three checks separate the two.

Measure event loop delay

perf_hooks.monitorEventLoopDelay() returns a histogram of how late the event loop runs relative to its expected schedule. Enable it at startup and log percentile(99) and max periodically, converting from nanoseconds to milliseconds. On a healthy server the values stay low and flat. If the p99 rises with traffic, or max reaches tens or hundreds of milliseconds, something is holding the main thread. Slow database calls do not do this, because awaiting I/O yields the loop. Synchronous code does not yield.

Compare a cheap endpoint

Under production-like load, measure p99 latency on an endpoint that does almost no work, such as a health check that returns a constant. If its latency tracks the latency of your heavy routes, requests are waiting behind each other on the same thread. If the cheap endpoint stays fast while one route is slow, the slowdown is local to that route, which usually points to a downstream dependency.

Profile the CPU

Start the process with --cpu-prof, or attach Chrome DevTools via --inspect, and record while under load. Look for wide, deep stacks of your own code: JSON parsing of large payloads, image resizing, hashing, compression, or template rendering.

This article is for teams who see all three signals together. If you don't, worker threads will not fix your latency.

Copy// Log event loop delay alongside a cheap endpoint to check for main-thread blocking.
const http = require('node:http');
const { monitorEventLoopDelay } = require('node:perf_hooks');

// Sample loop delay every 20 ms. Values are recorded in nanoseconds.
const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();

const toMs = (ns) => (ns / 1e6).toFixed(1);

// Every 10 s, log p50/p99/max and reset so each window is independent.
setInterval(() => {
  console.log(
    `event-loop-delay p50=${toMs(histogram.percentile(50))}ms ` +
    `p99=${toMs(histogram.percentile(99))}ms max=${toMs(histogram.max)}ms`
  );
  histogram.reset();
}, 10_000).unref(); // Don't keep the process alive just for metrics.

http.createServer((req, res) => {
  if (req.url === '/healthz') {
    // Cheap endpoint: does no work. If its p99 tracks the heavy routes under
    // load, requests are queuing behind synchronous code on the main thread.
    res.writeHead(200, { 'content-type': 'text/plain' });
    return res.end('ok');
  }
  // ... your real routes ...
  res.writeHead(404);
  res.end();
}).listen(3000);

// To find the hot path, run under load with a CPU profile:
//   node --cpu-prof server.js
// Stop the process cleanly, then open the generated .cpuprofile file in Chrome DevTools.

A minimal HTTP server that logs event loop delay percentiles from perf_hooks.monitorEventLoopDelay and exposes a constant /healthz endpoint for latency comparison, with a note on capturing a CPU profile via --cpu-prof.

What worker threads are and why async I/O does not help CPU-bound Node code

Marking a function async does not move its work anywhere. Node's async APIs help when an operation spends its time waiting on a socket, a disk, or a timer: the main thread hands the wait to the operating system or the libuv thread pool and keeps serving other requests. A loop that parses, hashes, or resizes in JavaScript is not waiting on anything. It runs on the main thread until it finishes. Wrapping it in a Promise only delays when it starts, and while it runs, no other callback on the event loop can execute.

The worker_threads module gives you additional JavaScript threads. Each worker has its own V8 isolate, its own heap, and its own event loop. Workers do not share module state, the module cache, in-memory caches, or open connections with the main thread. A database pool created in the main thread does not exist inside a worker. If the worker needs one, it has to create its own. The two sides communicate by message passing, and values are copied using the structured clone algorithm unless you explicitly transfer them.

A minimal round trip has four parts:

• Construct a Worker with the path to a script.
• Send input with worker.postMessage(). The worker receives it through parentPort and replies with parentPort.postMessage().
• The main thread receives the result in a 'message' listener.
• Uncaught exceptions inside the worker are emitted as 'error' on the main thread. 'exit' always fires when the worker stops and reports an exit code, which tells you whether a request ended without producing a result.

Copy// sum-worker.js: runs as the main thread and as the worker
const { Worker, isMainThread, parentPort } = require('node:worker_threads');

if (isMainThread) {
  // Start a worker that runs this same file in its own isolate and event loop.
  const worker = new Worker(__filename);
  let gotResult = false;

  worker.on('message', (result) => {
    gotResult = true;
    console.log('sum of squares:', result);
  });

  // Uncaught exceptions thrown inside the worker arrive here.
  worker.on('error', (err) => {
    console.error('worker failed:', err.message);
  });

  // 'exit' always fires. Check it to catch requests that never got a result.
  worker.on('exit', (code) => {
    if (!gotResult) console.error(`worker exited with code ${code} and no result`);
  });

  // The input is copied with structured clone. The worker cannot see main-thread state.
  worker.postMessage({ n: 50_000_000 });
} else {
  // Worker side: a single request. Once this listener is removed, the port no
  // longer keeps the worker alive, so it exits after replying.
  parentPort.once('message', ({ n }) => {
    if (!Number.isSafeInteger(n) || n < 0) throw new Error('n must be a non-negative integer');
    // CPU-bound loop. It blocks this worker's thread only, not the main event loop.
    let total = 0;
    for (let i = 0; i < n; i++) total += i * i;
    parentPort.postMessage(total);
  });
}

A single-file round trip in which the main thread starts a Worker, sends input with postMessage, receives the result on 'message', and uses 'error' and 'exit' to detect a worker that failed or stopped without replying.

Why a worker per request fails and how a bounded pool fixes it

Those lifecycle events make a worker easy to wrap in a promise, which makes it tempting to call new Worker() inside every request handler. That pattern breaks in three ways under real traffic.

Startup cost

Each worker is a separate V8 isolate with its own event loop and module graph. Creating one means initializing the isolate, bootstrapping the Node.js runtime, and loading and compiling your worker script and its imports. This work happens before your task code runs, and you pay it on every request. For short tasks, startup can take longer than the computation itself.

Memory per isolate

Every isolate has its own heap, its own copies of loaded modules, and its own JIT-compiled code. Twenty concurrent requests means twenty copies of all of it. You can cap heap size per worker with the resourceLimits option, but that only bounds each copy. It does not limit how many copies exist.

Unbounded concurrency

Nothing limits how many workers exist at once. When workers outnumber cores, the operating system time-slices them, so every request slows down together. Memory grows with the request rate until the container hits its limit and the kernel OOM-kills the process. That takes down every in-flight request, not only the new ones.

The pooled shape

A pool creates a fixed set of workers at startup and keeps them alive. The main event loop accepts a request, then places the task in a queue with a maximum length. Idle workers take tasks from that queue. Backpressure applies at admission: when the queue is full, the server rejects the request immediately, for example with a 503, instead of accepting work it cannot finish.

Copy HTTP requests
      |
      v
+--------------------------------------------------------------+
| Main thread (event loop)                                     |
|                                                              |
|  admit --> queue full? ---yes---> reject immediately (503)   |
|               |                   (backpressure)             |
|              no                                              |
|               v                                              |
|  +--------------------------------------------+              |
|  | Bounded task queue (max length N)          |              |
|  | [task] [task] [task] [    ] [    ]         |              |
|  +--------------------------------------------+              |
|       |               |               |          ^           |
+-------|---------------|---------------|----------|-----------+
        | idle worker   | pulls next    | task     | result
        v               v               v          | message
  +------------+  +------------+  +------------+   |
  | Worker 1   |  | Worker 2   |  | Worker K   |   |
  | long-lived |  | long-lived |  | long-lived |   |
  +------------+  +------------+  +------------+   |
        |               |               |          |
        +---------------+---------------+----------+

  K workers are created once at startup and reused for every task

The main event loop admits requests into a queue capped at N tasks and rejects them with a 503 when it is full, while a fixed set of K long-lived workers pulls tasks from the queue and posts results back to the main thread.

Implementing a Piscina pool with backpressure and cancellation

The task module is a file that exports one function. Piscina loads it in every worker through the filename option. Keep it pure: input in, result out, no shared state. A worker can be torn down at any moment, so nothing it holds should matter.

Sizing and admission

Base maxThreads on the CPUs the process can actually use, not the host's core count. Leave one for the main thread's event loop. Set minThreads equal to maxThreads so a traffic spike does not also pay worker startup cost. Set maxQueue to a small multiple of the thread count. When the queue is full, run() rejects immediately. Catch that specific rejection in the route handler and return 503 with a Retry-After header. Reserve 429 for per-client limits, because a full pool is a server-wide condition, not something the caller did.

Cancellation

Pass an AbortSignal to run(). Create one controller per request and abort it when the response closes before it finishes, which means the client went away, or when the request deadline expires. If the task is still queued, Piscina drops it. If it is already running, Piscina terminates that worker and starts a replacement. Termination is the only reliable way to stop synchronous JavaScript.

Failure handling

That same mechanism covers infinite loops. Every task needs a deadline signal, because a runaway task otherwise holds its thread forever. An error thrown inside the task rejects that one run() call. An uncaught exception from async code kills the worker, and Piscina replaces it. Alert on the replacement rate, not only on queue depth.

Copyimport { createServer } from 'node:http';
import { availableParallelism } from 'node:os';
import Piscina from 'piscina';

// availableParallelism() honors CPU affinity but not cgroup quotas; override in containers.
const threads = Math.max(1, Number(process.env.WORKERS) || availableParallelism() - 1);
const pool = new Piscina({
  filename: new URL('./hash-task.js', import.meta.url).href, // exports one pure function
  minThreads: threads, // pre-warm: spikes don't pay worker startup
  maxThreads: threads,
  maxQueue: threads * 4, // small, bounded backlog; beyond this run() rejects at once
});

createServer(async (req, res) => {
  const client = new AbortController();
  // 'close' before the response finishes means the client went away.
  res.on('close', () => { if (!res.writableFinished) client.abort(); });
  // Deadline also kills runaway or infinite-loop tasks: Piscina terminates and replaces the worker.
  const signal = AbortSignal.any([client.signal, AbortSignal.timeout(2000)]);

  try {
    const result = await pool.run({ input: 'example-payload' }, { signal });
    res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify(result));
  } catch (err) {
    if (res.destroyed) return; // client is gone, nobody to answer
    if (err.message === 'Task queue is at limit') {
      // Server-wide saturation: 503, not 429.
      res.writeHead(503, { 'Retry-After': '1' }).end();
    } else if (err.name === 'AbortError') {
      res.writeHead(504).end(); // deadline expired
    } else {
      res.writeHead(500).end(); // error thrown inside the task
    }
  }
}).listen(3000);

A Piscina pool sized from available CPUs with a bounded queue, returning 503 when full and cancelling tasks via AbortSignal on client disconnect or deadline.

Data transfer cost and pool sizing inside containers

Every message to a worker pays for structured clone, which serializes and copies the payload on one thread and deserializes it on the other. For large JSON-like objects, that copy can cost more than the work itself.

Illustrative estimate (assumed numbers, not a benchmark). Assume a task computes for 0.3 ms, the request payload takes 0.4 ms to clone, and the result takes 0.1 ms to clone back.

Overhead ratio = (clone_in + clone_out) / compute = (0.4 + 0.1) / 0.3 ≈ 1.7

In this case the worker spends more time moving data than computing. Batch small tasks, pass binary data as transferables (Piscina's run() accepts a transfer list), or keep the work inline.

For sizing, os.cpus() reports host cores, not the container's CPU quota. A pod limited to 2 CPUs on a 64-core node will be throttled by a 64-thread pool. Count every thread that competes for that quota:

• The libuv threadpool (default 4) for crypto, zlib and fs.
• Each cluster process, which runs its own pool.
• Other replicas packed onto the same node, if you size from node capacity.

Each worker also has its own V8 heap. Set resourceLimits so that (workers × per-worker heap) + main heap stays under the container memory limit.

Observability for the worker pool

A pool hides CPU pressure from the event loop, so event loop delay alone no longer shows saturation. Keep sampling it with perf_hooks.monitorEventLoopDelay, which should stay flat once work is offloaded. If it rises, something CPU-heavy is still on the main thread, or serialization of large payloads is costing more than expected. Alongside it, track five pool signals:

• Queue depth: tasks waiting for a free worker. Alert on sustained depth, not brief spikes.
• Wait time vs. run time: record both per task, as histograms, not averages. If wait time rises while run time stays flat, the pool is saturated and you need more capacity or less admitted work. If both rise together, the tasks themselves got slower, from larger inputs, a regression, or CPU throttling in the container.
• Worker utilization: the fraction of time workers spend running tasks. If it stays near 100% while queue depth grows, the pool is undersized. If it is low while latency is high, look at transfer cost or the main thread.
• Failures and timeouts: count these separately. A timeout means your cancellation path ran, and a crash means a worker died and was replaced.
• Rejected requests: backpressure working as designed. This is the metric that tells you about capacity before users see timeouts.

Debugging is harder across the thread boundary. An error thrown in a worker reaches the caller as a serialized copy, so its stack shows the worker's frames, not the request handler that queued the task. Also, CPU profiles and tracing spans recorded in a worker carry no request identity unless you pass one in. Include a request or trace ID in every task payload, and use it to tag the worker's logs and spans.

When to avoid worker threads: alternatives, cost and engineering time

Worker threads fit a narrow band of work: CPU-bound, short enough to finish within the request's latency budget, and safe to lose if the process dies. Outside that band, another option is usually cheaper.

Illustrative estimate (assumed numbers, not a benchmark): main-thread time saved per task = t_task − t_dispatch, where t_dispatch is the time the main thread spends serialising the task and handling the reply. Assume t_dispatch = 0.2 ms. For a 0.3 ms task, offloading saves 0.1 ms and adds latency and complexity. For a 40 ms task, it saves 39.8 ms. Measure both values in your own service before you decide.

Workers also cost engineering time. Each one loads its own modules and warms its own JIT. Each one holds a separate heap. Errors cross a message boundary and lose their context, and tests need a real pool or a fake that behaves like one. Even so, a bounded pool is often the cheaper choice overall. You can defer extracting a service until scaling actually requires it, and overload behaves predictably: tasks queue up to a fixed limit, then get rejected, so the server does not degrade without warning.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to node.js worker threads, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.