Service Mesh

Updated September 30, 20262 min read

In short

A service mesh is an infrastructure layer that manages traffic between microservices, adding encryption, retries, routing, and monitoring without code changes.

What is a service mesh?

A service mesh is a dedicated layer that handles communication between the services of a microservices application. Instead of every service implementing its own logic for secure connections, retries, timeouts, and metrics, the mesh provides those features in a consistent way for all of them. Developers can focus on business logic while the platform team controls how services talk to each other.

Most meshes work by placing a small proxy next to each service, called a sidecar, or by running shared proxies on each node in a so-called sidecarless or ambient mode. All traffic between services flows through these proxies, which form the data plane. A separate control plane gives them configuration and certificates, so it can turn on mutual TLS (mTLS), where both sides of a connection prove their identity, and apply rules such as sending 10% of traffic to a new version.

Think of it as air traffic control for your services: each plane, or request, still flies to its destination, but a shared system handles routing, safety rules, and tracking for all flights. A service mesh is most useful in large Kubernetes environments with dozens or hundreds of services, where consistent security and observability are hard to achieve by hand. For a small application, it can add more complexity and resource overhead than it saves.

A service mesh is often confused with an API gateway. An API gateway sits at the edge and manages north-south traffic, meaning requests coming from outside clients into the system, while a service mesh manages east-west traffic between internal services. Many systems use both: the gateway for public entry and the mesh for communication inside the cluster.

Key takeaways

A service mesh manages service-to-service traffic in a microservices system.
Proxies form the data plane, and a control plane configures them.
It provides mutual TLS, retries, timeouts, traffic splitting, and telemetry.
These features are added without changing application code.
An API gateway handles traffic entering the system, while a mesh handles traffic inside it.

Example

Setting a timeout for calls to an internal service (Gateway API for service meshes)yaml

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: orders
spec:
  parentRefs:
    - group: ""
      kind: Service
      name: orders
  rules:
    - timeouts:
        request: 2s
      backendRefs:
        - name: orders
          port: 8080

Readers ask

Do I need a service mesh?

Probably not if you run only a few services, because a mesh adds operational complexity and extra resource use. It becomes valuable when you have many services and need consistent encryption, traffic control, and observability across all of them.

What is a sidecar in a service mesh?

A sidecar is a small proxy container that runs next to each application container in the same pod and intercepts its network traffic. Newer sidecarless designs move this proxy to a shared component on each node to reduce overhead.

What is the difference between a service mesh and an API gateway?

An API gateway manages traffic coming into the system from outside clients, handling things like authentication and rate limiting at the edge. A service mesh manages the traffic between services inside the system.