Courses › System Design › Key Technologies

Nginx & Envoy

Reverse proxies: TLS termination, L4 vs L7 routing, timeouts, retries, circuit breaking and service meshes.

Nginx and Envoy are the two proxies you're most likely to name in an interview. They sit in front of your services as reverse proxies, load balancers and API gateways. Nginx is the classic, battle-tested web server and edge proxy. Envoy is the modern, dynamically configured proxy behind most service meshes. HAProxy is the other common name; it plays the same role as Nginx for load balancing.

What a reverse proxy does

[Clients] -> [Nginx / Envoy: TLS, routing, limits] -> [Service A pods]
[Nginx / Envoy: TLS, routing, limits] -> [Service B pods]
  • TLS termination — decrypt HTTPS once at the edge, so services can speak plain HTTP inside the network.
  • Routing — send /api/users/* to the users service and /static/* to a cache, by host, path or header.
  • Load balancing — round robin, least connections, consistent hashing on a header or cookie.
  • Health checks — stop sending traffic to unhealthy backends.
  • Rate limiting and connection limits — protect backends from abuse and bursts. See Rate Limiters.
  • Caching and compression — serve repeated responses and gzip/brotli content.
  • Long-lived connections — proxy WebSockets and HTTP/2 or gRPC streams.

L4 vs L7

Both Nginx and Envoy do both. A common setup is a cloud L4 load balancer in front of a fleet of L7 proxies.

Nginx

  • Event-driven with a few worker processes, each handling thousands of connections — very efficient for static files and proxying.
  • Configured with a file; changes need a reload (graceful, without dropping connections).
  • Often used as the Kubernetes ingress controller and as a static-content server.

Envoy

  • Built at Lyft for microservices. Configured dynamically through APIs (xDS), so a control plane can update routes and endpoints in real time without reloads.
  • First-class support for HTTP/2, gRPC, retries with budgets, timeouts, circuit breaking, outlier detection (ejecting a misbehaving instance) and detailed metrics and tracing.
  • The data plane of service meshes like Istio: an Envoy sidecar next to each service handles mutual TLS, retries and telemetry, so application code doesn't have to.

[!TIP] Name the resilience features Saying "the sidecar proxy enforces a 300 ms timeout, retries idempotent requests once with a retry budget, and trips a circuit breaker if the payments service's error rate spikes" shows you know how to stop one slow dependency from cascading into an outage.

When to name which

  • Nginx — edge reverse proxy, static content, simple routing and rate limiting, ingress
  • Envoy — service-to-service traffic, gRPC, dynamic config, service mesh, advanced resilience
  • Managed options — AWS ALB/NLB, API Gateway or Cloud Load Balancing when you'd rather not run proxies at all

[!INTERVIEW] Chat service with WebSockets "Clients connect through an L4 load balancer to an Envoy fleet that terminates TLS and routes WebSocket upgrades to chat servers using consistent hashing on the user ID, so reconnects tend to land on the same server. Envoy's health checks and outlier detection remove a failing chat server, and clients reconnect with backoff." See Real-time Updates.