Service Mesh Architecture

Difficulty: Senior Level | Companies: AWS, Google, Microsoft, Netflix, Uber

What is a Service Mesh

A service mesh provides infrastructure-level networking between microservices. It handles service discovery, load balancing, encryption, and observability without application code changes.

ℹ️

Service mesh moves networking concerns from application code to infrastructure. Your services focus on business logic; the mesh handles everything else.

Mesh Architecture

Architecture Diagram

                    ┌─────────────────────────────────────┐
                    │           Control Plane             │
                    │  ┌─────────────────────────────┐   │
                    │  │  Istiod / App Mesh           │   │
                    │  │  • Configuration             │   │
                    │  │  • Certificate Authority     │   │
                    │  │  • Traffic Management        │   │
                    │  └─────────────────────────────┘   │
                    └───────────────┬─────────────────────┘
                                    │ xDS API
              ┌─────────────────────┼─────────────────────┐
              │                     │                     │
     ┌────────▼───────┐   ┌────────▼───────┐   ┌────────▼───────┐
     │   Service A    │   │   Service B    │   │   Service C    │
     │ ┌───────────┐  │   │ ┌───────────┐  │   │ ┌───────────┐  │
     │ │  App Pod  │  │   │ │  App Pod  │  │   │ │  App Pod  │  │
     │ └───────────┘  │   │ └───────────┘  │   │ └───────────┘  │
     │ ┌───────────┐  │   │ ┌───────────┐  │   │ ┌───────────┐  │
     │ │ Envoy     │  │◀─▶│ │ Envoy     │  │◀─▶│ │ Envoy     │  │
     │ │ Proxy     │  │   │ │ Proxy     │  │   │ │ Proxy     │  │
     │ └───────────┘  │   │ └───────────┘  │   │ └───────────┘  │
     └────────────────┘   └────────────────┘   └────────────────┘

Pattern 1: Istio Traffic Management

Route traffic with canary deployments and A/B testing.

# VirtualService for canary deployment
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: order-service
spec:
  hosts:
    - order-service
  http:
    - match:
        - headers:
            x-canary:
              exact: "true"
      route:
        - destination:
            host: order-service
            subset: canary
    - route:
        - destination:
            host: order-service
            subset: stable
          weight: 90
        - destination:
            host: order-service
            subset: canary
          weight: 10
      timeout: 10s
      retries:
        attempts: 3
        perTryTimeout: 2s
        retryOn: "5xx,reset,connect-failure"
---
# DestinationRule for subsets
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: order-service
spec:
  host: order-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        h2UpgradePolicy: DEFAULT
        http1MaxPendingRequests: 100
        http2MaxRequests: 1000
        maxRequestsPerConnection: 10
    loadBalancer:
      simple: LEAST_REQUEST
  subsets:
    - name: stable
      labels:
        version: v1
    - name: canary
      labels:
        version: v2

ℹ️

Istio's traffic management enables progressive delivery without application code changes. Route based on headers, weights, or any request attribute.

Pattern 2: Mutual TLS Between Services

Encrypt all service-to-service communication automatically.

# PeerAuthentication - enforce mTLS
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT
---
# AuthorizationPolicy - service-to-service access control
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: order-service-policy
  namespace: production
spec:
  selector:
    matchLabels:
      app: order-service
  rules:
    - from:
        - source:
            principals:
              - "cluster.local/ns/production/sa/api-gateway"
              - "cluster.local/ns/production/sa/payment-service"
      to:
        - operation:
            methods: ["GET", "POST"]
            paths: ["/api/v1/*"]
    - from:
        - source:
            principals:
              - "cluster.local/ns/production/sa/inventory-service"
      to:
        - operation:
            methods: ["GET"]
            paths: ["/api/v1/inventory/*"]

Pattern 3: Fault Injection for Resilience Testing

Test service resilience by injecting faults.

# Fault injection for chaos engineering
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: order-service-chaos
spec:
  hosts:
    - order-service
  http:
    - fault:
        delay:
          percentage:
            value: 10
          fixedDelay: 5s
        abort:
          percentage:
            value: 5
          httpStatus: 503
      route:
        - destination:
            host: order-service
            subset: stable

Pattern 4: Observability with Envoy Metrics

Collect distributed tracing and metrics from sidecar proxies.

# Prometheus ServiceMonitor for Istio metrics
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: istio-mesh
  labels:
    release: istio
spec:
  selector:
    matchExpressions:
      - key: istio
        operator: In
        values: [pilot]
  namespaceSelector:
    matchNames:
      - istio-system
  endpoints:
    - port: http-monitoring
      interval: 15s
---
# Jaeger tracing configuration
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    enableTracing: true
    defaultConfig:
      tracing:
        sampling: 100.0
        openCensusAgent:
          address: "jaeger-collector.observability:55678"
          context:
            - W3C_TRACE_CONTEXT
            - B3

⚠️

High tracing sampling rates generate massive data. Start with 1% sampling in production and increase for debugging specific issues.

Pattern 5: AWS App Mesh Configuration

AWS-native service mesh alternative to Istio.

# App Mesh mesh definition
Mesh:
  Type: AWS::AppMesh::Mesh
  Properties:
    MeshName: production-mesh
    Spec:
      EgressFilter:
        Type: ALLOW_ALL
---
# Virtual Node definition
VirtualNode:
  Type: AWS::AppMesh::VirtualNode
  Properties:
    MeshName: !Ref Mesh
    VirtualNodeName: order-service
    Spec:
      Listeners:
        - PortMapping:
            Port: 8080
            Protocol: http
          HealthCheck:
            Protocol: http
            Path: /health
            HealthyThreshold: 2
            UnhealthyThreshold: 3
            TimeoutMillis: 5000
            IntervalMillis: 15000
      Backends:
        - VirtualService:
            VirtualServiceName: !GetAtt InventoryService.VirtualServiceName
      ServiceDiscovery:
        AWSCloudMap:
          NamespaceName: production.local
          ServiceName: order-service
---
# Traffic policy
VirtualService:
  Type: AWS::AppMesh::VirtualService
  Properties:
    MeshName: !Ref Mesh
    VirtualServiceName: order-service.production.local
    Spec:
      Provider:
        VirtualNode:
          VirtualNodeName: !Ref VirtualNode

Service Mesh Comparison

Feature	Istio	AWS App Mesh	Linkerd
Complexity	High	Medium	Low
mTLS	Automatic	Manual	Automatic
Traffic Management	Advanced	Basic	Moderate
Observability	Full	CloudWatch	Full
Overhead	~10ms	~5ms	~2ms

Follow-Up Questions

How do you decide between Istio and AWS App Mesh for a multi-cloud Kubernetes deployment?
What strategies would you use to migrate from direct service calls to service mesh incrementally?
How do you handle service mesh overhead in latency-sensitive applications?