Find where pipeline time actually goes

Most slow pipelines aren't slow because of the tests. The time usually goes to repeated work: dependencies downloaded on every run, toolchains rebuilt from scratch, and full test suites triggered by a README edit. Redundant runs add more, such as three pushes to a branch producing three complete pipelines when only the last one matters. Speed-up work means removing this repetition first, then running the remaining work in parallel. Fixing these patterns is cheaper and safer than rewriting tests.

Read the run before changing it

Open a workflow run and expand each job. The log lists every step with its duration, which shows whether time is spent installing packages, compiling, or testing. Compare the run's total time with the sum of its job durations. If jobs waited before starting, that is queue time: the job was waiting for a runner. Caching won't reduce queue time. It comes from runner availability and concurrency limits, which is a separate problem.

For a wider view, use the GitHub Actions metrics available to organizations and repositories. They help you rank workflows by how much they run and how long they take, so you can start with the most expensive ones.

Illustrative estimate (assumed numbers, not a benchmark): daily cost = runs per day × minutes per run. With 200 runs × 4 minutes of dependency installation, that is 800 runner-minutes per day spent on one repeated step.

Backend and platform engineers who own shared CI benefit most, because a fix to a shared workflow applies to every team that uses it. Start optimizing when engineers regularly wait on CI or when minute usage keeps rising.

How actions/cache works: keys, restore-keys and scope

Each cache entry is stored under a key string. When the step runs, it first looks for an exact match on key. If it finds one, it restores the files and sets the cache-hit output to true. If it doesn't, it tries each restore-keys entry in the order you listed them, treating each one as a prefix. When several entries share a prefix, the most recently created one wins. List the most specific prefix first and the broadest last.

Build the primary key from your lockfile, for example hashFiles('**/package-lock.json') combined with runner.os. Any dependency change then produces a new key, so you never restore a stale tree as if it were current. A prefix match still restores a close predecessor, and the package manager only has to fetch the difference.

Entries cannot be overwritten. On a miss, the action saves a new entry under the primary key in its post-job step, and only if the job succeeded. This keeps a failed or half-finished install from being stored and handed to every later run.

Caches are scoped to the branch that created them:

feature/login run, lookup order:
  1. feature/login   key, then restore-keys
  2. main (default)  key, then restore-keys
  x  feature/search  never visible

main ──► readable by every branch
 ├── feature/login   (isolated)
 └── feature/search  (isolated)

Each repository has a cache storage limit. When usage goes over it, the least recently used entries are evicted first. Many short-lived keys from busy feature branches can push out the default-branch caches that every new branch relies on.

name: ci

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Cache npm download cache
        id: npm-cache
        uses: actions/cache@v4
        with:
          path: ~/.npm
          # Exact key: changes whenever any lockfile changes.
          key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
          # Prefix fallbacks, most specific first. When several entries
          # match a prefix, the most recently created one is restored.
          restore-keys: |
            ${{ runner.os }}-npm-

      - name: Report cache result
        # 'true' only on an exact key match. A restore-keys match leaves it
        # unset, and the entry is saved under the new key after the job succeeds.
        run: echo "exact hit: ${{ steps.npm-cache.outputs.cache-hit }}"

      # npm ci always runs. A warm ~/.npm means it fetches only what changed.
      - run: npm ci
      - run: npm test

This workflow builds an npm cache key from the OS and a hash of package-lock.json, adds an OS-only prefix as a restore-keys fallback, and prints the cache-hit output so you can tell an exact match from a fallback restore.

Setup-action caching versus a custom cache step

For most dependency installs, the cheapest fix is the caching that the official setup actions already include. In actions/setup-node, set the cache input to npm, yarn or pnpm. In actions/setup-python, use pip, pipenv or poetry. Recent major versions of actions/setup-go turn caching on by default. Each action builds the key from your lockfile (package-lock.json, poetry.lock, go.sum), so you don't write or maintain key logic yourself. Note that these actions cache the package manager's download store, not node_modules or a virtualenv. The install step still runs, but it reads packages from local disk instead of the network.

A custom actions/cache step is still worth it for things the setup actions don't handle:

Option What it caches Who builds the key Best for
Setup-action cache input Package manager download store The action, from the lockfile npm/pip/Go dependencies
Custom actions/cache Any path you list You Build outputs, compiler caches (ccache, sccache), downloaded tool binaries
No cache Nothing Nobody Small or poorly compressible inputs

When to skip caching

Every cache restore downloads and decompresses an archive. If the dependency tree is small, or the contents don't compress well, restoring can take as long as a clean install. In that case the cache only adds complexity. Watch for two key problems:

  • Keys that are too broad. A key that leaves out the lockfile hash will restore stale dependencies.
  • Keys that never hit. A key that includes a timestamp or commit SHA misses on every run, so you pay the upload cost each time and never get a restore.

Check the cache hit rate in the job logs. Also make sure every workflow passes on a cold cache: an evicted entry should only make a run slower, never make it fail.

name: ci
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      # Built-in caching: the action caches npm's download store (~/.npm),
      # keyed on package-lock.json. You don't write any key logic.
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      # Custom cache for a build output the setup action doesn't handle.
      # The key includes the lockfile and source hashes, so a dependency or
      # code change produces a new entry instead of restoring stale output.
      # restore-keys lets a partial match seed an incremental build.
      - uses: actions/cache@v4
        with:
          path: .next/cache
          key: next-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-${{ hashFiles('src/**') }}
          restore-keys: |
            next-${{ runner.os }}-${{ hashFiles('package-lock.json') }}-

      # Install and build always run. A cache miss makes them slower but
      # never breaks them, so the workflow still passes on a cold cache.
      - run: npm ci
      - run: npm run build

A Node workflow that uses setup-node's built-in npm cache for dependencies and a custom actions/cache step only for the Next.js build cache, with keys scoped to the lockfile and source hashes so they neither restore stale output nor miss on every run.

Cancel superseded runs with concurrency groups

Even with good caching, the cheapest run is one that never finishes. When someone pushes three commits to a pull request in quick succession, the first two runs check code nobody will merge. A concurrency block stops that waste. Set the group key to the workflow name plus the ref, for example ${{ github.workflow }}-${{ github.ref }}, and set cancel-in-progress: true. When a new run enters the same group, GitHub cancels the older one. On a busy repository, this saves CI minutes and frees runners for work that still matters. Including the workflow name stops two different workflows on the same branch from cancelling each other.

The same setting is harmful on the default branch. There, every commit usually needs its own result for bisecting, release tagging or deployment. cancel-in-progress accepts an expression, so make it conditional, for example true only when github.ref is not your default branch, or only for pull_request events.

Deploy jobs need a separate rule. Put them in their own group, such as one per environment, and leave cancel-in-progress off. A new deploy then waits for the current one instead of killing it. Keep in mind that a group holds one running job and one pending job. A newer queued deploy replaces an older one that is still waiting, which is usually what you want.

Cancelling a deploy is risky because the runner stops mid-step. A rollout can end with some hosts on the new version and others on the old one. A schema migration can stop after some statements and before others. Release publishing can push artifacts without the tag or changelog that goes with them. Cancellation does not roll anything back, so the environment is left in whatever partial state it reached.

name: ci

on:
  pull_request:
  push:
    branches: [main]

# Workflow-level group: one active run per workflow + ref.
# Superseded PR runs are cancelled; runs on main are never cancelled
# mid-flight. They queue, though a newer pending run still replaces an
# older pending one in the same group.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: make test

  deploy:
    needs: test
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    # Separate group per environment: a new deploy waits for the current
    # one instead of killing it halfway through a rollout or migration.
    concurrency:
      group: deploy-production
      cancel-in-progress: false
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy.sh production

A workflow that cancels superseded pull request runs, lets default-branch runs finish, and puts the production deploy job in its own non-cancelling concurrency group so deploys queue instead of being interrupted.

Design the matrix to parallelize useful work

A matrix multiplies jobs, so every axis you add multiplies cost. Start from the combinations that catch real bugs. Use exclude to drop pairs that add little, such as an old runtime on a secondary OS. Use include to add one-off entries, like a nightly toolchain, without expanding the whole grid.

strategy.fail-fast defaults to true. When one matrix job fails, GitHub cancels the rest still in progress or queued. That saves minutes on pull requests, where any failure blocks the merge. Turn it off when you need the full picture, for example in a compatibility matrix where knowing that only one platform breaks is the useful signal. max-parallel caps how many matrix jobs run at once. Set it when jobs hit a shared database, a staging environment or a rate-limited registry, so parallelism doesn't turn into flaky failures.

Sharding a long test suite

Define an axis such as shard: [1, 2, 3, 4] with a fixed total. Each job passes its index and the total to the test runner, which runs only its slice. This needs a runner that supports deterministic partitioning, or a script that splits test files by a stable hash.

The trade-offs are real:

  • Total minutes can go up. Every shard repeats checkout, toolchain setup and cache restore. Four shards pay that overhead four times.
  • Parallelism has a ceiling. Your plan's concurrent job limit and available runners cap how many shards actually run at once. Shards beyond that cap wait in the queue.
  • Some suites cannot be split safely. Tests that depend on run order or share state will fail when split apart.
  • The slowest shard sets wall-clock time. Balance shards by recorded duration rather than file count.
name: ci
on: [pull_request]

jobs:
  compat:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false        # report every platform, not just the first failure
      matrix:
        os: [ubuntu-latest, windows-latest]
        node: [18, 20, 22]
        exclude:
          - os: windows-latest  # old runtime on a secondary OS adds little signal
            node: 18
        include:
          - os: ubuntu-latest   # one-off extra entry without growing the grid
            node: 23
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "${{ matrix.node }}", cache: npm }
      - run: npm ci
      - run: npm run build

  test:
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 2         # tests share a staging DB; limit concurrent load
      matrix:
        shard: [1, 2, 3, 4]   # keep the total in sync with --shard below
    steps:
      # Each shard repeats checkout, setup and cache restore
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      # Jest (28+) runs only this shard's slice of test files
      - run: npx jest --shard=${{ matrix.shard }}/4

A compatibility matrix that uses exclude, include and fail-fast: false, alongside a test job that splits a Jest suite into four shards with max-parallel limiting load on a shared resource.

Skip work entirely: path filters and conditional jobs

The fastest job is one that never starts. Every workflow trigger is a candidate for this.

Filter triggers by path

On push and pull_request events, paths runs the workflow only when a matching file changes. paths-ignore skips it when every changed file matches. You cannot use both for the same event. If you need to include and exclude in one rule, use paths with ! negation patterns. A docs-only change does not need a 20-minute integration suite. A change under services/billing/ does not need the frontend build.

Gate expensive jobs on cheap ones

Inside a workflow, needs creates ordering. If lint or unit tests fail, the jobs that depend on them are skipped, so they consume no runner time. Add if conditions to narrow this further. For example, run deployment previews only on pull requests, or run the full browser matrix only on the default branch. The goal is to spend seconds on checks that fail often before spending minutes on checks that rarely do.

The required-check trap

Branch protection does not treat skips consistently:

  • If path filtering skips an entire workflow, its required checks stay Pending. The pull request cannot merge.
  • If an if condition skips a job, the job reports Success. A change can merge without anything having tested it.

To avoid both problems, add one final aggregator job and make it the only required check. Give it needs on every job and if: always() so it always runs. Have it fail explicitly when any needed job's result is failure or cancelled.

Then move path logic out of workflow triggers and into job-level conditions inside that workflow. The workflow always runs, so the aggregator always reports a status.

Impact on CI minutes, cost and engineering time

Pricing is the last piece to work out before changing workflows. On GitHub-hosted runners, the per-minute rate depends on the runner OS. Linux is cheapest, and Windows and macOS minutes cost more, so the same saving is worth more on those runners. Check the current rates in the GitHub Actions documentation. Each job's time is rounded up to the whole minute, which matters once you split work into many short jobs.

Illustrative estimate (assumed numbers, not a benchmark). Assume each PR gets 4 pushes and 2 of those runs are superseded halfway. Each run takes 12 minutes, of which 4 are dependency setup. A cache hit cuts setup to 1 minute, and the hit rate is 80%.

  • Baseline: 4 × 12 = 48 billed minutes per PR.
  • Cancellation: 2 × 12 + 2 × 6 = 36 minutes.
  • Caching: expected setup is 0.8 × 1 + 0.2 × 4 = 1.6 minutes, so a run takes 9.6 minutes. Combined with cancellation: 2 × 9.6 + 2 × 4.8 = 28.8 minutes.
  • Sharding the 8 minutes of work across 3 jobs: per job, ⌈1.6 + 8/3⌉ = 5 billed minutes. That is 15 billed minutes per run instead of 9.6, but wall-clock time drops to about 4.3 minutes.

General formula: billed = Σ over jobs of ⌈setup + work/N⌉.

Technique Saves Costs Use when
Cancellation Minutes spent on superseded runs Lost results for intermediate pushes Branches get frequent pushes
Caching Setup time on every run Cache storage, key maintenance Dependency install is slow and stable
Sharding Wall-clock time Extra billed minutes from repeated setup and rounding Feedback latency matters more than spend

Engineering time usually outweighs billed minutes. A 4-minute signal keeps the author working on the change instead of switching tasks while waiting. Fewer queued runs also leave runner capacity free for everyone else's PRs.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to speed up github actions, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.