Monolith CI by Module: Selective Tests From a Dependency Graph

What module-based CI in a monolith repository is, and who it fits

Monolith CI by module keeps all the code in one repository but stops treating every change as a reason to rebuild everything. The pipeline looks at which files a change touches, maps them to modules, and then uses an explicit, checked-in dependency graph to find every module that depends on those modules. Only that set gets built and tested. A change to the billing module runs billing's tests plus the tests of anything that imports billing. It leaves the unrelated search indexer alone.

Teams adopt this for three reasons. First, build time drops for most changes. Second, runner cost falls because fewer jobs run. Third, and usually most important, feedback latency improves: a developer waiting eight minutes instead of forty stays on the task. They get this without the polyrepo coordination cost. Polyrepos bring version pinning between internal packages, multi-repo pull requests for one logical change, and drift between repos that are supposed to be compatible. Here, an atomic commit across modules is still possible, and so is a single full build when you want one.

Who it fits

The approach pays off when three things are true:

• The repository has several reasonably independent modules.
• The full pipeline is long enough to hurt.
• Enough contributors push changes that queue time compounds.

A small repo with a five-minute pipeline gains little.

Plain path filters are not a substitute. They know which directories changed. They do not know who depends on those directories. When a shared library changes, a path filter runs that library's tests and skips its consumers. Nothing reports the skipped tests, so the build looks green when it has not checked what matters. The dependency graph exists to close that gap.

Defining module boundaries and a checked-in dependency file

The graph lives in a file you check in and review, not in a script that guesses.

Start with the directory layout you already have. A module is a directory whose contents build and test as a unit, such as a service, a shared library or a worker. Every tracked path must map to exactly one module or to the global list. Make CI fail on unmapped paths, because an unowned file is a change that triggers nothing.

Keep one file at the repository root that lists each module's path prefixes and the modules it depends on. Inferring edges from import statements looks cheaper but misses coupling that never shows up in an import. A checked-in file also means every graph change goes through code review, with an author and a diff.

Implicit coupling needs a decision for each kind:

Some files affect every module at once. Changes to any of these force a full build:

• Root build configuration
• Dependency lockfiles
• CI pipeline definitions
• Shared tooling and build scripts
• Base container images

Treating these as global keeps the per-module edge lists from growing to include everything.

Copy# modules.yaml: checked in at the repo root and read by CI.
# Every tracked path must match a module prefix or a global pattern;
# CI fails on any path that matches neither.

global:
  # A change to any of these forces a full build of every module.
  - BUILD.root.toml          # root build configuration
  - "**/package-lock.json"   # dependency lockfiles
  - "**/go.sum"
  - .ci/**                   # CI pipeline definitions
  - tools/**                 # shared tooling and build scripts
  - docker/base/**           # base container images

modules:
  db-schema:                 # migrations; consumers depend on it explicitly
    paths: [db/migrations/]
    depends_on: []
  config:                    # runtime config shared by some services
    paths: [config/shared/]
    depends_on: []
  contracts:                 # protobuf/OpenAPI definitions
    paths: [api/proto/, api/openapi/]
    depends_on: []
  libs-common:
    paths: [libs/common/]
    depends_on: []
  orders-svc:
    paths: [services/orders/]
    # Queries the orders tables and serves the contract it produces.
    depends_on: [libs-common, db-schema, contracts, config]
  billing-worker:
    paths: [workers/billing/]
    # Shares the orders tables with orders-svc but no code with it.
    depends_on: [libs-common, db-schema, contracts]
  web-client:
    # Generated client code lives here; the edge points at its source.
    paths: [web/]
    depends_on: [contracts]

Shows an example checked-in dependency file that maps directory prefixes to modules. It records implicit coupling as explicit edges to schema, config and contract modules, and lists the global files that force a full build.

How selection works: from changed files to affected modules

The selection step turns a diff into a set of modules. Each stage is ordered so that when a step is uncertain, it falls back to running more tests, not fewer:

1. Find the base. Run git merge-base origin/main HEAD and diff from that commit, for example git diff --name-only <base> HEAD. Diffing against HEAD~1 only sees the last commit of a multi-commit branch. Diffing against the current tip of main picks up other people's merged changes as if they were yours.
2. Ensure history exists. CI checkouts are often shallow, so the merge base may not have been fetched. Deepen the fetch with git fetch --deepen=<n> or --unshallow until merge-base succeeds. If it still fails, run a full build. That can happen after a force-push rewrote the base or when the target branch is missing. A guessed base produces a wrong file set with no error, so it is not safe to proceed.
3. Check globals. If any changed path matches the global list from the previous section, stop and build everything.
4. Map files to owners. Assign each path to the module with the longest matching path prefix. If a file matches no module, treat that as a full-build trigger. Skipping it would hide gaps in ownership.
5. Expand reverse dependencies. Invert the dependency file and walk it breadth-first from the owning modules. A change to shared/auth then selects every direct and transitive consumer, because those consumers are where a breaking change actually fails.

The output is a sorted module list, which the pipeline consumes in the next section. The script shown below implements these five steps and includes a test for each fallback path.

Copyimport collections, json, subprocess, sys

GLOBALS = ("WORKSPACE", ".ci/", "toolchain/")  # any hit forces a full build
FULL = None  # sentinel meaning: build everything

def git(*args):
    return subprocess.run(["git", *args], capture_output=True, text=True)

def merge_base(target="origin/main", attempts=5):
    # Shallow clones may lack the base; deepen a few times, then give up.
    for _ in range(attempts):
        r = git("merge-base", target, "HEAD")
        if r.returncode == 0:
            return r.stdout.strip()
        if git("fetch", "--deepen=200", "origin", "main").returncode != 0:
            break
    return None  # missing branch or rewritten history: never guess

def select(changed, modules, deps):
    if any(p.startswith(g) for p in changed for g in GLOBALS):
        return FULL
    owners = set()
    for p in changed:  # longest prefix wins; unowned files force a full build
        m = max((m for m in modules if p.startswith(m + "/")), key=len, default=None)
        if m is None:
            return FULL
        owners.add(m)
    rdeps = collections.defaultdict(set)  # invert: dependency -> consumers
    for mod, needs in deps.items():
        for d in needs:
            rdeps[d].add(mod)
    seen, queue = set(owners), collections.deque(owners)
    while queue:  # BFS picks up direct and transitive consumers
        for consumer in rdeps[queue.popleft()]:
            if consumer not in seen:
                seen.add(consumer)
                queue.append(consumer)
    return sorted(seen)

if __name__ == "__main__":
    deps = json.load(open("deps.json"))  # {"services/api": ["shared/auth"], ...}
    base = merge_base()
    diff = git("diff", "--name-only", base, "HEAD") if base else None
    ok = diff is not None and diff.returncode == 0
    result = select(diff.stdout.split(), list(deps), deps) if ok else FULL
    print("ALL" if result is FULL else "\n".join(result))

A selection script that diffs against the merge base, deepens shallow clones, falls back to a full build on globals, unowned files or a missing base, and expands owners through reverse dependencies.

Wiring selection into the pipeline and handling required checks

The pipeline has three stages. A small select job checks out the repository with enough history to compute the diff against the merge base, runs the selection script, and publishes the sorted module list as a job output, usually serialized as JSON. Keep this job fast and dependency-light, because every pull request waits on it.

Downstream, a single matrix job expands over that list, or there is one job per module, each conditioned on its name appearing in the output. A matrix keeps configuration short. Per-module jobs are easier when modules need different runners or toolchains. If the list is empty, the matrix must be skipped explicitly, because many CI systems treat an empty matrix as an error.

Branch protection is the trap. A required check that never runs stays pending, and the merge blocks. Do not mark per-module jobs as required. Instead, add an aggregate ci-result job that depends on everything, runs unconditionally, and fails if any upstream job failed or was cancelled. Make that job the only required check. Treat "skipped" as success only when select itself succeeded.

Native path filters still have a role as a coarse gate. For example, they can skip the whole pipeline when only documentation changed. They match file globs against the diff, though, and have no notion of the dependency graph. A change to a shared library cannot trigger its transitive dependents unless you hand-maintain globs that duplicate the dependency file and drift from it.

Skipped jobs also affect reporting:

• Test-history dashboards show gaps or misleading "flaky" patterns for skipped modules.
• Coverage reports drop sharply on partial runs. Compare coverage per module, or compute it only on full runs.

Copyname: ci
on:
  pull_request:
    # Coarse gate only: if a docs-only PR skips the whole workflow, ci-result never reports.
    # A required ci-result check would then stay pending, so make it required only if that case is handled.
    paths-ignore: ['docs/**']
jobs:
  select:
    runs-on: ubuntu-latest
    outputs:
      modules: ${{ steps.affected.outputs.modules }}
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }  # full history so the merge base is available
      - id: affected
        env: { BASE: "origin/${{ github.base_ref }}" }
        # Script walks the dependency graph and prints a sorted JSON array, e.g. ["billing","shared-auth"]
        run: echo "modules=$(./ci/select-modules.sh "$BASE")" >> "$GITHUB_OUTPUT"
  test:
    needs: select
    if: needs.select.outputs.modules != '[]'  # skip explicitly; an empty matrix is an error
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        module: ${{ fromJSON(needs.select.outputs.modules) }}
    steps:
      - uses: actions/checkout@v4
      - env: { MODULE: "${{ matrix.module }}" }
        run: ./ci/test-module.sh "$MODULE"
  ci-result:  # the only required check in branch protection
    needs: [select, test]
    if: always()  # run even when test was skipped, failed, or cancelled
    runs-on: ubuntu-latest
    steps:
      - run: |
          # Selection must have succeeded; only then is a skipped test job acceptable.
          [ "${{ needs.select.result }}" = "success" ] || exit 1
          case "${{ needs.test.result }}" in success|skipped) ;; *) exit 1 ;; esac

A GitHub Actions workflow in which a select job emits the affected modules as JSON and a matrix tests each one. An always-running ci-result job is the single required check, and the comments note the pending-check risk when paths-ignore skips the whole workflow.

Keeping a safety net for what selective CI misses

Selective CI works only as well as the dependency file is accurate, and that file will drift. Code gains an import nobody declared, a shared fixture starts serving a second module, or a config file affects behavior through a path the graph doesn't model. The fix is to keep running everything, just not on every pull request.

Run the full suite on the main branch after each merge, or on batches of merges if that queue grows too long, and also on a schedule such as nightly. Then alert specifically when the full run fails on a commit whose selective run passed. That pattern points to a miss in selection, not an ordinary bug. Treat each one as a missing edge: find which module's change broke which other module's tests, add that dependency to the checked-in file, and link the fix to the incident so the reason is recorded.

This has a real cost. Post-merge discovery means main can be broken for a while, other engineers may branch from a bad commit, and reverting is harder once later merges stack on top. Teams that cannot accept this should run the full suite in a merge queue before code lands.

The selection script is now critical infrastructure. Give it unit tests that cover known change-to-module mappings, assign it a named owner, and make it fail closed. If it crashes, can't parse the dependency file, or sees a path it doesn't recognize, it should select every module. A slow build is recoverable. A silently skipped test is not.

When to avoid it, and the alternatives

Selective CI only pays off when most pull requests stay inside one module. Skip it in these cases:

• Most changes touch shared code or root config. If a typical PR edits a common library, the lockfile, or the base build config, the selector resolves to "everything" nearly every time. You carry the script's complexity and get no speedup.
• The module graph is tangled or unknown. If modules import each other freely, the dependency file is guesswork. Either it is too broad and selects everything, or it is too narrow and misses breakages.
• The full suite is already fast. If the whole run finishes in a few minutes, even a large percentage cut saves little. It is not worth a new way for CI to be wrong.
• No one can own the selection script. A stale dependency file silently skips tests. Without an owner, the tool loses trust, and then people either rerun everything or ignore red builds.

Impact on cost and engineering time, measured with your own data

Before changing anything, export several weeks of pipeline history from your CI system and record four numbers per pull request: median wall-clock duration, p90 duration, total runner minutes, and queue time. The median shows the typical experience. The p90 shows whether the slow runs that block merges actually improved. Runner minutes track cost. Queue time shows whether extra parallel jobs are now waiting on capacity. Collect the same metrics over a comparable window after rollout. Avoid holiday weeks and release crunches, because both distort the numbers.

Savings depend on how often a change touches shared code and triggers the full suite. With s as that share, F as full-suite runner minutes, and P as a typical selective run:

Expected minutes per PR = s × F + (1 − s) × P

Illustrative estimate (assumed numbers, not a benchmark): F = 40, P = 10.

Subtract the offsetting costs from the saved figure:

• Parallel jobs: they add per-job setup minutes and can increase queue time.
• Cache maintenance: someone has to keep caches valid and fix them when they break.
• Dependency-file upkeep: review time on the checked-in dependency file adds up.
• Post-merge failures: hours spent investigating breakages that selection let through.

Track that last item explicitly, because each one can cancel out many small wins.

Saved runner minutes and saved engineering time are different numbers. Only the reduction in wall-clock time that an author actually spends waiting counts as engineering time. Measure the time from PR open to merge before and after rollout instead of assuming that every saved minute was reclaimed.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to monolith ci by module, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.