Why does copying the whole Nx monorepo into a Docker image break layer caching?

Most teams start with a Dockerfile that treats the Nx workspace as one project. It sets WORKDIR /app and runs COPY . . to bring in the entire repository. Next it runs npm ci against the root package.json and lockfile, then npx nx build orders-api, and finally starts the built output. This works, and it is the baseline the rest of this tutorial improves on.

The problem comes from how Docker caches layers. Each instruction produces a layer, and for COPY the cache key includes a checksum of every copied file. When any file in the build context changes, the COPY . . layer is invalidated, and so is every layer after it. That includes the dependency install. A one-line edit in apps/billing-worker forces a full npm ci when you build orders-api, even though orders-api never touches that code.

The root lockfile makes this worse. It describes the dependencies of every app and library in the workspace. When someone adds a charting library for the frontend, the lockfile changes, and every backend service's install layer rebuilds.

The resulting image also ships the whole workspace's node_modules. It includes test runners, build tooling, and packages only other services import. A small HTTP service ends up carrying dependencies it never loads, which makes images larger, slows pushes and pulls, and adds packages to vulnerability scans that are irrelevant to that service.

# Naive baseline: treats the whole Nx workspace as one build unit.
FROM node:20-alpine

WORKDIR /app

# Copies every app, lib, and the root lockfile into the image.
# The cache key for this layer covers every file in the build context,
# so an edit anywhere in the repo (e.g. apps/billing-worker) invalidates it.
COPY . .

# Installs dependencies for the ENTIRE workspace from the root lockfile.
# Because it follows COPY . ., this layer is rebuilt whenever any file changes,
# and it pulls in packages only other apps or the build tooling need.
RUN npm ci

# Builds one service inside the image, using the full workspace toolchain.
RUN npx nx build orders-api

# Runtime still carries the full source tree and the full node_modules.
ENV NODE_ENV=production
EXPOSE 3000
CMD ["node", "dist/apps/orders-api/main.js"]

The naive baseline Dockerfile: because COPY . . comes before npm ci, any change in the workspace invalidates the dependency install for orders-api, and the final image ships the entire workspace's node_modules.

What does generatePackageJson do and how does the pruned lockfile work?

generatePackageJson is a build option on Nx's bundling executors. When it is enabled, Nx reads the project graph, which it builds by analyzing the import statements in your source files, and collects the external npm packages the project reaches, including through the internal libraries it depends on. It then writes a package.json into the project's output directory. That file lists only those packages, with versions taken from the root package.json.

A version range alone would let npm install inside the image resolve different versions than the ones you tested. To prevent this, Nx also writes a lockfile in your package manager's format, pruned from the root lockfile. The pruned file keeps the entries for the selected packages and their transitive dependencies and drops the rest. The image therefore installs the same versions that the workspace resolved.

root lockfile ─┐
               ├──> nx build <service>
project graph ─┘          │
                          v
              dist/apps/<service>/
                ├── main.js        (bundle)
                ├── package.json   (only imported deps)
                └── <lockfile>     (pruned from root)
                          │
                          v
              Docker build context
                          │
                          v
     deps stage: copy manifest + lockfile, install
                          │
                          v
     runtime stage: copy node_modules + bundle

The recommended setup runs Nx on the host or CI runner, not inside the Dockerfile. Nx's task cache and affected detection only work when Nx can see the whole workspace and its cache directory. Docker then receives only dist/apps/<service>, which keeps the build context small. Because this folder contains no other services, changes to them cannot invalidate this service's image layers.

  HOST / CI RUNNER (full workspace, Nx task cache, affected detection)
 +--------------------------------------------------------------------+
 |  +---------------+      +---------------+                          |
 |  | root lockfile |      | project graph |                          |
 |  +-------+-------+      +-------+-------+                          |
 |          |                      |                                  |
 |          +----------+-----------+                                  |
 |                     v                                              |
 |          +---------------------+                                   |
 |          | nx build <service>  |  generatePackageJson: true        |
 |          +----------+----------+                                   |
 |                     v                                              |
 |          +--------------------------------------+                  |
 |          | dist/apps/<service>/                 |                  |
 |          |   main.js       bundle               |                  |
 |          |   package.json  only imported deps   |                  |
 |          |   <lockfile>    pruned from root     |                  |
 |          +------------------+-------------------+                  |
 +-----------------------------|--------------------------------------+
                               v  build context = dist folder only
 +--------------------------------------------------------------------+
 |  DOCKER BUILD                                                      |
 |  +-----------------------------------+                             |
 |  | deps stage: copy package.json +   |  cached until deps change   |
 |  | lockfile, install                 |                             |
 |  +-----------------+-----------------+                             |
 |                    v                                               |
 |  +-----------------------------------+                             |
 |  | runtime stage: node_modules+bundle|                             |
 |  +-----------------------------------+                             |
 +--------------------------------------------------------------------+

Nx runs on the host or CI runner to emit a bundle, a trimmed package.json and a pruned lockfile; Docker receives only that dist folder and installs dependencies in a cacheable deps stage.

How do you enable generatePackageJson and verify the dist output?

Open the service's project.json and find the build target. Under its options, set generatePackageJson to true. Put it in options rather than in a single configuration so that development and production builds both emit the manifest. If they don't, the Dockerfile can work locally and fail in CI. The option belongs to the executor, not to Nx itself, so it only works if your build executor implements it. Bundler-based executors such as the webpack and esbuild ones have supported it, but the option name, its behavior, and which lockfile formats get pruned (npm, Yarn, pnpm) have changed between plugin releases. Check the schema of the executor you have installed instead of assuming it matches what you read elsewhere.

Run the build with npx nx build <service>, then inspect the output with ls -la dist/apps/<service>. You should see the bundle, a package.json, and a lockfile in your package manager's format sitting next to it. If the lockfile is missing, the image will resolve versions at install time, and the pruning gives you nothing.

Read the manifest with cat dist/apps/<service>/package.json, or run jq '.dependencies' dist/apps/<service>/package.json to see only the dependencies. The list should contain only packages this service imports at runtime. If you find another service's database driver or a build-only tool in it, the usual causes are a dynamic require the dependency graph can't trace or a shared library that pulls in more than it needs. Fix that before you write the Dockerfile.

When the output looks right, pin the exact versions of Nx and its plugins in the root package.json. That way an upgrade can't silently change what gets pruned.

#!/usr/bin/env bash
# Build one service and check what generatePackageJson produced.
# Assumes project.json -> targets.build.options has "generatePackageJson": true
# and that your installed executor's schema supports that option.
set -euo pipefail

SERVICE=orders-api            # placeholder service name
OUT="dist/apps/${SERVICE}"

# Record the Nx and plugin versions this result depends on.
npx nx report

# Build the service; the executor writes the bundle and manifest to $OUT.
npx nx build "${SERVICE}"

# Expect the bundle, package.json, and a lockfile side by side.
ls -la "${OUT}"

# Fail early if no lockfile was emitted; without it the image
# resolves versions at install time and pruning gains nothing.
if ! ls "${OUT}"/package-lock.json "${OUT}"/yarn.lock "${OUT}"/pnpm-lock.yaml >/dev/null 2>&1; then
  echo "No lockfile in ${OUT}; check executor support for your package manager" >&2
  exit 1
fi

# Show only runtime dependencies. Every entry should be something this
# service imports; another service's driver or a build tool means a
# dynamic require or an over-broad shared library.
jq '.dependencies' "${OUT}/package.json"

# Build-time packages should not appear in the generated manifest.
jq '.devDependencies // "none"' "${OUT}/package.json"

Builds one service with generatePackageJson enabled, records the installed Nx versions, confirms a lockfile was emitted next to the generated package.json, and prints the manifest's dependencies for review.

How should you order Dockerfile layers so dependency installs stay cached?

The Dockerfile has two stages, and the order of COPY instructions inside them is what decides whether the cache helps you.

Deps stage

Start from the same base image you will run in production. Copy only the generated package.json and the pruned lockfile from the service's dist folder, then run a frozen, production-only install: npm ci --omit=dev, pnpm install --frozen-lockfile --prod, or the strict equivalent for your package manager. Docker computes a cache key for each COPY from the checksums of the files it copies. If those two files are byte-identical to the previous build, the install layer is reused. That holds no matter how much application code changed, because no code has entered the image yet.

Run the install here, on the runtime base image, rather than copying a node_modules built on your laptop or CI host. Native modules then compile or download prebuilt binaries for the image's OS, libc (glibc vs. musl) and CPU architecture. A host-built module can fail to load at runtime for any of those reasons.

Runtime stage

Use a slim image. Copy node_modules from the deps stage first, then the bundled application, so a code-only change invalidates just that final layer. Switch to a non-root user (official Node images ship a node user) and set the start command to run the bundle's entry file.

Build context

Keep the build context small. Point it at the service's dist folder, or add a .dockerignore that excludes the repository root's node_modules, .git and other services. This makes context uploads fast and keeps unrelated files out of any COPY.

# Build context: dist/apps/api (generated package.json, pruned lockfile, bundle)
# e.g. docker build -f apps/api/Dockerfile dist/apps/api

# --- deps stage: same base image as runtime so native modules match OS/libc/arch
FROM node:20-bookworm-slim AS deps
WORKDIR /app

# Copy ONLY the manifest and pruned lockfile. The cache key for this layer
# depends on these two files, so code-only changes reuse the install below.
COPY package.json package-lock.json ./

# Frozen, production-only install; fails if the lockfile is out of sync
RUN npm ci --omit=dev

# --- runtime stage
FROM node:20-bookworm-slim AS runtime
WORKDIR /app
ENV NODE_ENV=production

# Dependencies first: this layer changes only when the lockfile changes
COPY --from=deps --chown=node:node /app/node_modules ./node_modules

# Application bundle last: a code change invalidates only this layer
COPY --chown=node:node package.json main.js ./

# Official Node images ship a non-root 'node' user
USER node

CMD ["node", "main.js"]

A two-stage Dockerfile that installs production dependencies from only package.json and the pruned lockfile, then copies node_modules before the bundle so code-only changes reuse the cached install layer.

How do you keep the Docker layer cache working on ephemeral CI runners?

An ephemeral runner starts with an empty Docker daemon, so the layers built in the last job are gone. Every build is a cold build unless you export the cache and import it again. BuildKit does this through docker buildx build with --cache-to and --cache-from. A registry backend (type=registry,ref=...) works on any CI system. On GitHub Actions you can use the type=gha backend instead. Use mode=max if you want intermediate stages, such as the dependency-install stage, exported as well as the final stage.

Give each service its own cache ref, for example registry/cache:orders-api, or its own scope with the gha backend. If all services share one ref, each push overwrites the previous one, and services evict each other's dependency layers.

In each app's project.json, add a docker target that runs the buildx command and declares dependsOn: ["build"]. The pruned package.json and lockfile then exist before Docker reads them. Then run nx affected -t docker in CI.

The three layers of caching each skip a different kind of work:

  • Affected detection skips services whose project graph did not change.
  • Nx task caching restores dist output without recompiling.
  • Docker layer caching skips npm ci when the pruned lockfile is unchanged.

The time saved comes from fewer full rebuilds and shorter CI queues, because jobs finish sooner and free up runners.

Cache is not free. Every import downloads layers and every export uploads them. A cache that is oversized, or scoped so broadly that it rarely matches, can take longer to transfer than a cold build takes to run. Measure both paths before you commit to one.

How do you validate that per-service caching actually works?

Run each check with BuildKit's plain progress output (docker build --progress=plain) so every step is printed with its cache status instead of being collapsed.

1. Source-only change

Edit a single source file in one service, rebuild it with Nx, then rebuild its image. The dependency install step should print CACHED, and only the layers that copy the compiled output should run. If the install step re-executes, a file that changes on every build is being copied before it. Typical culprits are a timestamp written into the generated package.json or a COPY of the whole dist folder placed above the install.

2. Dependency change scoped to one service

Bump a package that only service A imports, then rebuild both services. Diff the generated package.json and lockfile in each dist folder. Service B's files should be byte-identical to the previous build, and its install step should report CACHED. If B's lockfile changed too, the pruning is pulling in more of the root lockfile than B needs, and per-service isolation is lost.

3. Compare image contents

docker history <image> shows each layer's size and the instruction that created it. Compare it before and after a change to see which layers were rebuilt. To check what is actually installed, list node_modules inside the image with docker run --rm <image> ls node_modules. Packages from other services or dev tooling should not appear there.

4. Run the image

A successful build only proves the install step finished. It does not prove the app can start. Start the container, hit its health endpoint, and exercise code paths that load modules lazily: dynamic require/import calls, optional dependencies, native addons, and plugins loaded by name from config. Static analysis tends to miss these, so they are where pruning most often drops a package the app needs at runtime.

When should you avoid this approach, and what are the alternatives?

Pruning is only as good as the dependency graph Nx can see. Packages resolved by name at runtime, such as ORM database drivers, plugin systems and logging transports, are absent from that graph, and so are native peer dependencies. The image builds cleanly and then fails on first use. Declare these packages explicitly in the project's dependencies and run a smoke test against the built image, not just the dist folder.

Lockfile pruning support differs across package managers and Nx versions. A pruned lockfile that disagrees with the generated package.json fails a frozen install. Switching to a non-frozen install makes the build pass, but it lets the resolver choose new versions, which removes the reproducibility you pruned for. If you bundle with esbuild or webpack, most dependencies are inlined and only externals need installing, so the externals list becomes one more thing to keep correct.

Caching has limits too. A shared internal library that changes often still invalidates the code layer of every service that imports it. Per-service images also cost more to run: more Dockerfiles or a templated one, more tags, and a security scan per image. In return you get a smaller dependency footprint that matches what each service actually uses.

Option Cache behaviour Image size Hermeticity Setup cost Maintenance cost
Whole-repo install Any lockfile change busts every image Largest Medium Low Low
Host build + pruned lockfile Per-service dependency layer Small Depends on CI host Medium Medium
Pruned workspace built in Docker Per-service, includes build stage Small High High Medium–high

Choose the in-Docker build when you need strict reproducibility or isolation from the CI host.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nx docker build pruned lockfile, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.