Courses › System Design › Key Technologies

Kubernetes

Pods, deployments, services, probes, rolling updates and autoscaling.

Kubernetes (K8s) is the standard platform for running containers across a cluster of machines. You describe the state you want — "run 10 copies of the API, reachable at this address, restarted if they crash" — and Kubernetes keeps reality matching it. Interviews rarely ask you to design Kubernetes itself, but knowing its vocabulary helps you answer "how is this deployed, scaled and kept healthy?" crisply.

Core objects

How it works

[kubectl / CI: desired state] -> [API server] -> (etcd: cluster state)
[API server] -> [Scheduler: place pods on nodes]
[API server] -> [Controllers: reconcile desired vs actual]
[API server] -> [kubelet on each node: run containers]
  • All cluster state lives in etcd — see ZooKeeper & etcd.
  • Controllers run reconciliation loops: if a deployment wants 10 pods and 9 are running, start one more. This declarative, self-healing model is the key idea.
  • The scheduler places pods on nodes with enough CPU and memory, respecting constraints such as spreading replicas across zones.

Features worth mentioning in a design

  • Health probes — a *liveness* probe restarts a stuck container; a *readiness* probe removes a pod from load balancing until it can serve traffic.
  • Rolling updates — replace pods gradually with maxUnavailable and maxSurge, rolling back automatically if new pods fail readiness. Blue-green and canary releases are built on top.
  • Autoscaling — HPA scales pods; the cluster autoscaler adds and removes nodes.
  • Resource requests and limits — guarantee capacity per pod and stop a noisy neighbor from starving others.
  • Topology spread and pod anti-affinity — keep replicas in different zones so one zone outage doesn't take down the service.

[!TIP] Stateless on Kubernetes, state in managed services A common, defensible answer: run stateless services as Deployments behind Services with autoscaling, and keep databases, queues and caches in managed services (RDS, MSK, ElastiCache). Running stateful systems on StatefulSets is possible, but it's more operational work.

Packaging & deploying

Nobody hand-writes YAML for every environment. Manifests are usually templated and applied automatically:

  • Helm packages an app's manifests as a *chart*, with a *values* file per environment (replica counts, image tags, resource limits). helm upgrade installs a new *release*, and helm rollback returns to a previous one.
  • Kustomize works the other way round: it starts from plain base manifests and applies per-environment *overlays*, with no templating language.
  • GitOps (Argo CD, Flux) — a controller in the cluster watches a Git repo and keeps the cluster in sync with it. A deploy is a commit, a rollback is a revert, and drift is corrected automatically. CI builds and pushes the image, then updates the tag in Git. See CI/CD.

The infrastructure *under* the cluster — the cluster itself, VPCs, load balancers, managed databases, queues — is usually defined as infrastructure as code with Terraform (or Pulumi, CloudFormation). Environments become reproducible, changes are reviewed as a plan diff before they're applied, and standing up a second region means applying the same modules again. In an interview, one sentence is enough: "infra is in Terraform, apps are deployed through GitOps."

When it's overkill

  • A handful of services with steady traffic — a managed platform (Cloud Run, App Engine, ECS/Fargate) or serverless functions is simpler.
  • Small teams without people to operate the cluster.

[!INTERVIEW] Remote dev environments "Each devbox is a pod from a StatefulSet-like controller with its own persistent volume, scheduled onto a node pool sized for developer workloads. An idle controller scales stopped devboxes to zero after 30 minutes and keeps the volume, so restarting takes seconds. Network policies isolate each user's pod." See Design a Remote Devbox.