Courses › Low-Level Design › Problem Walkthroughs

Design a Circuit Breaker

A closed, open and half-open state machine with failure-rate windows, trial calls and a fake clock.

A circuit breaker protects a service from a failing dependency. When calls to the dependency keep failing, the breaker "opens" and fails new calls immediately, instead of letting every request wait for a timeout. After a cool-down it lets a few trial calls through, and closes again if they succeed. The idea is explained in Resilience Patterns; in an LLD round you implement it as a class.

It's a compact problem that combines a state machine, time and concurrency, so it's a common machine-coding question and a frequent follow-up to the rate limiter.

Requirements

  • Wrap any call: breaker.call(fn, *args) returns the result or raises.
  • Closed: calls go through. Count failures; after the threshold is reached, open the breaker.
  • Open: fail immediately with CircuitOpenError for a cool-down period.
  • Half-open: after the cool-down, allow a limited number of trial calls. If they succeed, close; if any fails, open again.
  • Configurable thresholds, and which exceptions count as failures.
  • Thread-safe, and testable without real waiting.

The state machine

(CLOSED) -> [failures reach threshold] -> (OPEN)
(OPEN) -> [cool-down elapsed] -> (HALF_OPEN)
(HALF_OPEN) -> [trial calls succeed] -> (CLOSED)
(HALF_OPEN) -> [any trial call fails] -> (OPEN)

Choose how failures are counted, and say which you chose:

The implementation below uses a sliding window of recent outcomes with a minimum call count, which is what production libraries do.

Implementation

import threading
import time
from collections import deque
from enum import Enum


class State(Enum):
    CLOSED = "closed"
    OPEN = "open"
    HALF_OPEN = "half_open"


class CircuitOpenError(Exception): ...


class CircuitBreaker:
    def __init__(self, failure_rate=0.5, window=10, min_calls=5, cooldown=30.0,
                 trial_calls=2, failure_types=(Exception,), clock=time.monotonic):
        self.failure_rate, self.min_calls = failure_rate, min_calls
        self.cooldown, self.trial_calls = cooldown, trial_calls
        self.failure_types, self.clock = failure_types, clock
        self.outcomes = deque(maxlen=window)        # True = failure
        self.state = State.CLOSED
        self.opened_at = 0.0
        self.trials_started = self.trials_ok = 0
        self.lock = threading.Lock()

    def call(self, fn, *args, **kwargs):
        self._before_call()                          # may raise CircuitOpenError
        try:
            result = fn(*args, **kwargs)
        except self.failure_types:
            self._on_result(failed=True)
            raise
        except BaseException:                        # not counted as a failure, but frees a trial slot
            self._on_result(failed=None)
            raise
        self._on_result(failed=False)
        return result

    def _before_call(self):
        with self.lock:
            if self.state is State.OPEN:
                if self.clock() - self.opened_at < self.cooldown:
                    raise CircuitOpenError("circuit open")
                self._transition(State.HALF_OPEN)
            if self.state is State.HALF_OPEN:
                if self.trials_started >= self.trial_calls:
                    raise CircuitOpenError("half-open: trial calls in progress")
                self.trials_started += 1

    def _on_result(self, failed):
        with self.lock:
            if self.state is State.HALF_OPEN:
                if failed:
                    self._transition(State.OPEN)
                elif failed is False:
                    self.trials_ok += 1
                    if self.trials_ok >= self.trial_calls:
                        self._transition(State.CLOSED)
                else:
                    self.trials_started -= 1
                return
            if self.state is State.CLOSED and failed is not None:
                self.outcomes.append(failed)
                if (len(self.outcomes) >= self.min_calls and
                        sum(self.outcomes) / len(self.outcomes) >= self.failure_rate):
                    self._transition(State.OPEN)

    def _transition(self, state):                     # called with the lock held
        self.state = state
        if state is State.OPEN:
            self.opened_at = self.clock()
        if state is State.HALF_OPEN:
            self.trials_started = self.trials_ok = 0
        if state is State.CLOSED:
            self.outcomes.clear()

A test with a fake clock walks through every transition without sleeping:

class FakeClock:
    def __init__(self):
        self.now = 0.0

    def __call__(self):
        return self.now


clock = FakeClock()
cb = CircuitBreaker(failure_rate=0.5, window=4, min_calls=4, cooldown=10,
                    trial_calls=2, clock=clock)


def ok():
    return "ok"


def boom():
    raise ConnectionError("down")


for fn in (ok, boom, ok, boom):                # 2 of 4 failed = 50%
    try:
        cb.call(fn)
    except ConnectionError:
        pass
print(cb.state)                                # State.OPEN

try:
    cb.call(ok)
except CircuitOpenError as e:
    print("rejected:", e)                      # fails fast, ok() never runs

clock.now = 10.5                               # cool-down over
print(cb.call(ok), cb.state)                   # ok State.HALF_OPEN
print(cb.call(ok), cb.state)                   # ok State.CLOSED

for fn in (boom, boom, boom, boom):
    try:
        cb.call(fn)
    except ConnectionError:
        pass
clock.now = 25
try:
    cb.call(boom)                              # the trial call fails
except ConnectionError:
    pass
print(cb.state)                                # State.OPEN again

Design decisions to explain

  • The lock is never held while calling fn. The dependency might take seconds; holding the lock would serialize every caller. The lock only protects the state changes before and after.
  • Half-open admits a fixed number of trial calls. Without a limit, the moment the cool-down ends, every waiting request would hit a dependency that may still be struggling.
  • Not every exception is a failure. A ValueError from bad input says nothing about the dependency's health. failure_types lets the caller decide. Uncounted exceptions still free up their trial slot.
  • A minimum number of calls stops one failure out of one call (100%) from opening the breaker.
  • An injected monotonic clock makes tests instant and protects against wall-clock changes.

[!INTERVIEW] Timeouts belong next to the breaker A breaker only helps if slow calls actually fail. Mention that each call needs a timeout, and that a timeout should count as a failure. Without one, a hanging dependency never trips the breaker.

Follow-ups

  • Fallbacks: call(fn, fallback=...) returns cached or default data while open, instead of raising.
  • Metrics and events: emit state changes to a listener (Observer) so dashboards and alerts can show when a breaker opens.
  • One breaker per dependency, or per dependency and endpoint: a registry keyed by name, similar to the rate limiter registry.
  • Slow-call rate: also open the breaker when too many calls are slower than a threshold, even if they succeed.
  • Combining with retries: retries go *inside* the breaker for a single logical call, or outside it with care. Retrying while the breaker is open defeats its purpose.
  • Several instances: each server keeps its own breaker. That's usually fine, and simpler than sharing state.

[!TIP] Practice Design a Circuit Breaker · Implement a Rate Limiter Class · Concurrency & Thread Safety