Service Mesh: Do You Actually Need One? A Practical Guide

What a service mesh is and how it works

A service mesh is a layer of infrastructure that handles service-to-service traffic so application code doesn't have to. It has two parts.

The data plane

The data plane is a set of proxies placed in the network path of every workload. In the sidecar model, each pod gets its own proxy container, and traffic redirection rules (typically set up by an init container or a CNI plugin) force all inbound and outbound connections through it. In node-level designs, a shared proxy per node handles traffic for the pods on that node. This reduces per-pod overhead, but more workloads share a single component. In both designs, the application opens a plain connection and the proxy does the rest.

The control plane

The control plane never touches requests. It watches the cluster for services, endpoints and policy objects, then pushes routing and security configuration to the proxies. It also acts as a certificate authority. It issues each workload a short-lived certificate tied to its identity, usually derived from the Kubernetes service account, and rotates the certificate automatically.

One request, end to end

Say orders calls payments:

1. The orders app sends a plain request to payments.
2. The client-side proxy intercepts it and picks a healthy endpoint. It applies any configured retries and timeouts.
3. It opens a mutually authenticated TLS connection, presenting the orders certificate.
4. The server-side proxy verifies that certificate and checks authorization policy, such as whether orders may call payments. It then forwards plaintext to the local app.

Both proxies emit metrics and trace spans for the call. Identity, encryption and policy all come from infrastructure, which is why they apply consistently regardless of language or framework.

Copy+--------------------------------------------------------------------------+
|                CONTROL PLANE (never on the request path)                 |
|  watches services, endpoints, policy -> pushes config to proxies         |
|  certificate authority -> issues and rotates workload certs              |
+--------------------------------------------------------------------------+
                 | routing config                          | routing config
                 | + orders cert                           | + payments cert
                 v                                         v
+--------------------------------+        +--------------------------------+
| Pod: orders                    |        | Pod: payments                  |
|   +------------+               |        |               +------------+   |
|   | orders app |               |        |               |payments app|   |
|   +------------+               |        |               +------------+   |
|          |  1 plain request    |        |  4 plaintext         ^         |
|          v                     |        |                      |         |
|   +------------------------+   |(3) mTLS|   +------------------------+   |
|   | client-side proxy      |-------------->| server-side proxy      |   |
|   | 2 pick endpoint, retry |   |        |   | 4 verify cert, authz   |   |
|   | timeouts; present cert |   |        |   | check policy           |   |
|   +------------------------+   |        |   +------------------------+   |
|                |               |        |                |               |
+----------------+---------------+        +----------------+---------------+
                 v                                         v
+--------------------------------------------------------------------------+
| TELEMETRY: both proxies emit metrics and trace spans for every call      |
+--------------------------------------------------------------------------+

The control plane pushes config and workload certificates to the proxies, and the proxies carry an orders-to-payments request over mTLS, handling routing and policy and emitting telemetry outside the application code.

What Kubernetes, libraries and your ingress already give you

Before adding that layer, compare it with what a standard cluster already does. Kubernetes provides service discovery through DNS, load balancing across a Service's pods, self-healing through liveness and readiness probes, and controlled rolling updates. A NetworkPolicy restricts which pods can talk to each other, but only if your CNI plugin enforces it. An ingress controller or Gateway API implementation can route by host, path or header and split traffic by weight for canaries at the edge. Resilience libraries add retries, timeouts and circuit breakers inside the process.

These features have limits. kube-proxy balances at L4, per connection. A long-lived HTTP/2 or gRPC connection stays pinned to one pod, so request-level balancing does not happen. NetworkPolicy matches on labels, namespaces and IP ranges, not on cryptographic workload identity, and it does not encrypt anything. Edge canaries do not apply to east-west calls. Libraries work, but each language stack has to implement and configure them separately.

If the right-hand column covers nothing you need, the mesh mostly adds operating cost.

mTLS between services, traffic control and observability: what you actually get

Identity and mTLS

The mesh's control plane acts as a certificate authority. Each workload gets a short-lived certificate tied to its identity, which on Kubernetes is usually its service account. The sidecar or node proxy rotates that certificate automatically, so no team handles key files or expiry dates. In Istio, permissive mode accepts both plaintext and mTLS. That lets you enrol services one at a time without breaking callers that have not joined the mesh yet. Strict mode rejects plaintext. Plan the switch from permissive to strict as a deliberate step. If you never make it, you have not actually enforced encryption.

Encryption alone does not restrict who can call a service. It proves the caller is some meshed workload. Authorization policy uses that proven identity to decide which workloads may call this one. For example, a rule might allow only the checkout service account to call payments, and only with POST. The policy example below shows both the mode setting and a rule like this.

Traffic control

Retries, timeouts, weighted canary splits (for example, 95/5) and mirroring are all set in configuration and enforced by the proxy. Mirroring sends a copy of live requests to a new version and discards its responses. Two cautions apply. Only retry idempotent calls, and keep retry budgets low, because stacked retries across several hops multiply load during an outage. Mirrored traffic can also write data twice if the shadow service has side effects.

Observability limits

The proxy reports the golden signals for every hop: latency, traffic, errors and saturation. It sees network calls only. It cannot see slow queries, lock contention or a hot loop inside your process. Distributed tracing also needs work in the application. Each service must copy incoming trace headers, such as traceparent, onto its outbound requests. Without that, spans from different hops are never joined into one trace. The propagation snippet below shows this step.

Copy# Mesh-wide default: accept plaintext and mTLS while services enrol.
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system   # root namespace, so this applies mesh-wide
spec:
  mtls:
    mode: PERMISSIVE
---
# The payments namespace has fully joined the mesh, so reject plaintext here.
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: strict-mtls
  namespace: payments
spec:
  mtls:
    mode: STRICT
---
# mTLS proves the caller's identity. This rule decides who may call.
# Once an ALLOW policy selects a workload, requests that match no rule are denied.
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: payments-allow-checkout
  namespace: payments
spec:
  selector:
    matchLabels:
      app: payments
  action: ALLOW
  rules:
  - from:
    - source:
        # SPIFFE-style identity taken from the checkout service account's cert
        principals: ["cluster.local/ns/shop/sa/checkout"]
    to:
    - operation:
        methods: ["POST"]

Istio policy that keeps the mesh permissive by default, enforces strict mTLS in the payments namespace, and lets only the checkout service account call payments, using POST only.

Istio vs Linkerd: choosing by the features you will use

Both projects deliver mTLS between services, retries, timeouts and per-request metrics. They differ in how much else they try to do, and that difference determines how much of your team's time each one consumes.

Istio aims to cover almost every traffic scenario: fine-grained routing, fault injection, egress control, multi-cluster topologies and extension points in the proxy. It is built on Envoy, a general-purpose proxy with a very large configuration model. Istio can run Envoy as a sidecar or in its ambient mode, which moves per-node L4 handling out of the pod and adds optional L7 proxies. That breadth helps if you need it. If you don't, it means more custom resources to learn, more interactions to debug and more upgrade notes to read.

Linkerd deliberately does less. It uses its own lightweight proxy, purpose-built for the mesh rather than configurable for any use, and it enables mTLS for meshed workloads with little setup. The trade is fewer knobs: if you later need advanced egress policy or proxy extensions, you may hit its limits.

Choose based on requirements you can name today. "We might need fault injection someday" does not justify carrying Istio's configuration surface now.

When a service mesh is worth it and when to avoid it

A service mesh earns its place when it solves a problem you already have and someone is ready to run it. The mesh sits in the request path of every call between services. When it breaks, everything breaks. So the decision depends on your organization as much as on the features.

Signals for and against

The language signal matters because a mesh replaces per-language library work. With five languages, keeping retry and mTLS behaviour identical in five codebases costs more than running proxies. With one language, a shared library does the same job with fewer moving parts.

Ownership is the signal teams most often skip. Without an owner, nobody upgrades the control plane on time, certificate expiry turns into an outage, and engineers learn to blame "the mesh" for every latency spike.

Who should use it

A mesh fits organizations that already run a platform team, operate many services written by independent teams, and face a concrete security or multi-cluster requirement. Smaller teams usually get more from NetworkPolicy, good client libraries and ingress-level TLS. They can revisit the decision when one of the signals in the "Adopt" column becomes true.

Trade-offs and impact on cost and engineering time

The control plane is another distributed system you operate. It needs upgrades, often on a cadence tied to your Kubernetes versions, and someone on call who understands it. If proxies and control plane drift apart in version, failures get harder to reason about.

Illustrative estimate (assumed numbers, not a benchmark). Assume 200 pods, 0.05 vCPU and 60 MiB per sidecar, and about 1 ms added per hop. Extra CPU = pods × CPU per proxy = 200 × 0.05 = 10 vCPU. Extra memory = 200 × 60 MiB ≈ 11.7 GiB. Added latency = hops × per-hop cost, so a 4-hop call adds about 4 ms. Measure these in your own environment, under your own traffic.

The offset is engineering time. You don't have to implement mTLS, retries and telemetry once per language and keep each version patched. You also get one enforceable policy for every service, where library-based approaches drift as each team upgrades on its own schedule.

An incremental adoption path

Before you install anything

Write down what success means and what triggers a rollback. Success might be p99 latency within an agreed margin of baseline, no rise in 5xx rates, and complete service-to-service telemetry for the pilot namespace. Rollback triggers should be just as specific, such as pods stuck in init, failed health checks, or a latency regression that lasts past one deploy cycle. If you set these criteria after rollout, you will end up rationalizing whatever happened.

Pilot one namespace

Pick a non-critical namespace and enable the mesh only there. In Istio, that means the istio-injection=enabled label plus a namespace-scoped PeerAuthentication in PERMISSIVE mode. Permissive mode accepts both plaintext and mTLS, so callers outside the mesh keep working. Roll the pods, then check three things:

• Telemetry: golden metrics match what your existing dashboards report.
• Failure behavior: killing an upstream fails the way you expect.
• Sidecar lifecycle: Jobs terminate cleanly, and apps don't start sending traffic before the proxy is ready.

Decide who owns resilience

Retries and timeouts must live in one layer. If the client library retries three times and the mesh also retries three times, a single failing call can turn into nine. Pick one owner and remove the other layer's settings.

Tighten, then expand

Once telemetry shows no plaintext traffic left, switch the namespace to STRICT mode. Then repeat the process one namespace at a time. Manage all mesh configuration through pull requests, validate it in CI, and test it in staging. A single bad policy can break traffic across the whole cluster, so treat changes to it with the same care as application code.

Copy# Stage 1: pilot namespace. Sidecar injection applies only here.
apiVersion: v1
kind: Namespace
metadata:
  name: reports-pilot
  labels:
    istio-injection: enabled   # new pods in this namespace get the Envoy sidecar
---
# Namespace-scoped policy. PERMISSIVE accepts both mTLS and plaintext,
# so callers that are not yet in the mesh keep working during the pilot.
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default                # 'default' applies to every workload in the namespace
  namespace: reports-pilot
spec:
  mtls:
    mode: PERMISSIVE
---
# Stage 2: commit this in a separate, reviewed PR once telemetry shows
# no plaintext traffic to this namespace. It has the same name, so applying
# it replaces the Stage 1 policy. To roll back, revert the PR and set the
# mode back to PERMISSIVE.
#
# apiVersion: security.istio.io/v1beta1
# kind: PeerAuthentication
# metadata:
#   name: default
#   namespace: reports-pilot
# spec:
#   mtls:
#     mode: STRICT             # reject plaintext; only mesh clients with mTLS can connect

Turns on Istio sidecar injection for a single pilot namespace, sets mTLS to PERMISSIVE for that namespace, and holds the STRICT policy as a separate step that can be applied or reverted once telemetry confirms no plaintext traffic remains.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to service mesh, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.