Service mesh & sidecars

A hotel booking platform has 60 services written in Java, Go, Python and Node. Every call between them should be encrypted and authenticated, have a timeout and a sensible retry, be spread across healthy instances, and show up in the metrics. Each team built those features with a library for its language:

The real cost is change. Fixing a retry bug means a release of four libraries and sixty services. Deploying library upgrades across a large service architecture can be incredibly painful. A service mesh takes the other route: put the networking in a separate process beside each service, the same process for every language, and upgrade it without touching application code.

In the classic design, every instance of every service gets a sidecar proxy in the same pod. Rules in the pod's network setup redirect all its inbound and outbound traffic through the proxy, so the application talks plain HTTP to what looks like the service it called, and never knows the proxy is there:

The proxies are the data plane: they carry every request. The control plane never touches requests; it configures the proxies. In Istio, the control plane (istiod) turns routing rules into Envoy configuration and pushes it to every sidecar at runtime, tracks service discovery, and acts as a certificate authority for the proxies' identities. The protocol Envoy uses to receive that configuration, xDS, has become a common language: gRPC can speak it directly.

Linkerd follows the same split with its own small proxy written in Rust, built only for the mesh rather than as a general proxy. Kubernetes now supports sidecars natively, stable since v1.33: a sidecar starts before the application container and, when the pod shuts down, is stopped only after the application has stopped, so the application never loses its network on the way in or out.

With the proxies in place, the platform team configures behavior once, for all 60 services:

Turning on mutual TLS (TLS in which both sides present a certificate, so each knows who the other is) in a running system has a migration path built in. Istio's PERMISSIVE mode lets a workload accept both mTLS and plain text while callers are still being moved into the mesh; once every caller has a proxy, STRICT mode accepts mutual TLS only. That makes "encrypt everything inside the data center" a configuration change rather than sixty code changes (Encryption in transit & at rest).

Per-request load balancing also fixes a problem plain Kubernetes Services have with long-lived connections: a connection-level balancer never moves an open connection to new instances, while a proxy chooses an instance for each request (Service discovery & registration). Traffic splitting lets the mesh run a canary without touching the deployment (Deployment strategies).

Nothing in the path is free. Each call now passes through two extra proxies, and each proxy needs CPU and memory. Istio publishes its own measurements: in Istio 1.24, at 1,000 requests a second with 1 KB payloads, one sidecar with two worker threads uses about 0.20 vCPU and 60 MB of memory. Memory grows with the amount of routing configuration the proxy holds.

At the hotel platform's 400 pods, memory alone adds up:

Two subtler costs catch teams out:

• Retries multiply. Istio retries a failed HTTP request twice by default, so up to three attempts per hop. If booking calls pricing, pricing calls rates, and rates calls inventory, a failing inventory can receive 3 × 3 × 3 = 27 attempts for one user request, more if the applications retry too. Set retries in one place and budget them (Timeouts, retries, backoff & jitter).
• Tracing still needs the application. Proxies send spans for every call, but they can't know which outgoing request belongs to which incoming one. Each service must copy the trace headers across:

And the mesh itself must be operated: upgraded, scaled, monitored and understood when it misbehaves. A wrong routing rule pushed by the control plane reaches every proxy, so mesh configuration deserves the same staged rollout as code (Feature flags & configuration).

The per-pod sidecar is where most of the cost lives, so meshes now offer alternatives:

• Ambient mode in Istio splits the job in two. A per-node proxy, ztunnel, written in Rust, handles only layer 4: mutual TLS, authentication, layer 4 authorization and telemetry, without parsing HTTP. Only where layer 7 features are needed, such as HTTP routing, retries or authorization by path, is a waypoint proxy added, running outside the application pods.
• Proxyless gRPC removes the proxy altogether for gRPC services: the gRPC library speaks xDS itself, receiving routing, load balancing and mTLS configuration from the same control plane.

Istio's own tests of proxyless gRPC found a marginal latency increase over plain gRPC, which it called a massive improvement over going through Envoy. The modes also mix: sidecar and ambient workloads can coexist in one mesh, so a platform can start with layer 4 encryption everywhere and add layer 7 features only for the services that need them.

A mesh handles traffic between services; the edge still needs a gateway for authentication of users, rate limits and public routing (Reverse proxies & API gateways). For the hotel platform, the checklist:

• Start in permissive mode, move every service into the mesh, then switch to strict mTLS.
• Write authorization policies by service identity, not by network address.
• Configure timeouts and retries in one place, and remove duplicate retries from application code.
• Forward trace headers in every service.
• Budget the proxies' CPU and memory, and consider ambient or proxyless modes where the overhead matters.
• Roll out mesh configuration in stages, like any other production change.