Courses › System Design › Question Breakdowns

Design a Payment System

Idempotency at every hop, payment state machines, a double-entry ledger and reconciliation.

Design the backend that takes money from buyers and pays sellers for a marketplace, using external payment processors (Stripe, Adyen, card networks) to actually move the money. Throughput is modest; the interview is about correctness: idempotency, state machines, a double-entry ledger and reconciliation with systems you don't control.

Requirements

Functional

  • Buyers can pay for an order with a card (via an external processor)
  • Sellers are paid out their share, minus fees, on a schedule
  • Support refunds
  • Every movement of money is recorded in an auditable ledger

Out of scope: building a card network, fraud models (mention a risk check), currency conversion, tax.

Non-functional

  • No money lost or duplicated — a retried request must never charge twice
  • Consistency over availability for the money path; it's fine to show "processing" for a while
  • Auditability: every balance can be explained by immutable entries
  • Scale: ~1M payments/day (~12/s average, ~100/s peak) — small, so correctness dominates
  • Security: card numbers never touch our servers (PCI scope)

[!TIP] The question behind the question "The processor timed out. Did the charge happen?" Interviewers want to hear that you can't know from the response, so your design must make retrying safe (idempotency) and must find out later (reconciliation, webhooks). Everything else follows from that.

Core Entities

  • Payment intent — ID, order, amount, currency, status, idempotency key, processor reference
  • Ledger entry — transaction ID, account, debit/credit, amount, created at (immutable)
  • Account — buyer funds, seller balance, platform fees, processor clearing
  • Payout — seller, amount, status, bank reference

API

POST /v1/payments
Idempotency-Key: 7d4c...
{ "orderId": "o_42", "amount": 5000, "currency": "USD", "paymentMethodToken": "pm_tok_..." }
→ 201 { "paymentId": "pay_9", "status": "processing" }

GET  /v1/payments/{paymentId}      → { "status": "succeeded" }
POST /v1/payments/{paymentId}/refunds   { "amount": 5000 }
POST /webhooks/processor           (processor → us: charge.succeeded, charge.failed, dispute.created …)

The client sends a payment method token produced by the processor's hosted fields / SDK, so raw card data never reaches us.

High-Level Design

buyer = client "Checkout" @0,0
api = service "Payment Service" @1,0
pdb = store "Payments DB\n(intents)" @1,1
exec = service "Payment Executor" @2,0
psp = service "External Processor\n(Stripe / Adyen)" @3,0
ledger = store "Ledger\n(double entry)" @2,1
wh = service "Webhook Handler" @3,1
recon = service "Reconciliation" @2,2
payout = service "Payout Service" @1,2
buyer -> api : pay
api -> pdb : create intent
api -> exec
exec -> psp : charge (idempotency key)
psp --> wh : async result
wh -> pdb
wh -> ledger : record entries
recon -> psp : settlement files
recon -> ledger : compare
payout -> ledger
  • Create intent: the payment service stores an intent (created) keyed by the client's idempotency key and returns its ID.
  • Execute: the executor calls the processor, passing the same idempotency key on every retry.
  • Result: the processor's response (or its later webhook) moves the intent to succeeded / failed, and successful charges write ledger entries.
  • Payouts read seller balances from the ledger and send bank transfers on a schedule.
  • Reconciliation compares our ledger with the processor's settlement reports every day.

Deep Dives

1. Idempotency end to end

  • Client → us: Idempotency-Key header. Store (key, request hash, response); a retry with the same key returns the stored response instead of creating a second payment. A unique constraint on the key makes concurrent duplicates fail fast.
  • Us → processor: pass a key derived from the payment ID. Processors dedup on it, so if our call timed out and we retry, they return the original result rather than charging again.

2. The payment state machine

created -> processing -> succeeded -> refunded
processing -> failed
processing -> requires_action

Every transition is a conditional update (UPDATE … SET status='succeeded' WHERE id=:id AND status='processing'), so a late webhook can't move a refunded payment back to succeeded, and two handlers can't apply the same transition twice.

Unknown outcomes: if the processor call times out, leave the intent in processing. A background job retries the call (safe, thanks to the key) or queries the processor's status API; the webhook usually arrives first anyway.

3. Double-entry ledger

Every money movement is a transaction made of entries whose debits and credits sum to zero:

  • Entries are append-only: a refund is a new transaction with reversed entries, never an edit.
  • A balance is the sum of an account's entries (with periodic snapshots for speed).
  • Write the ledger entries and the intent's status change in one database transaction (Postgres), so the ledger never disagrees with payment state.

[!WARNING] Don't store balances as a mutable number UPDATE accounts SET balance = balance + 45 loses the history that explains the number, and concurrent updates need careful locking. A ledger of immutable entries is both auditable and easier to reason about under concurrency.

4. Reconciliation

Your records and the processor's will drift: webhooks get lost, timeouts leave unknown states, the processor reverses a charge (chargeback). Every day, download the processor's settlement report and match it line by line with the ledger:

  • Matched: nothing to do.
  • In the processor, not in our ledger: a missed webhook — apply it.
  • In our ledger, not in the processor: investigate; flag for manual review.

This is the safety net that turns "probably right" into "provably right".

5. Processing webhooks safely

Webhooks arrive at least once and out of order. Verify the signature, store the event ID (dedup), enqueue it, and return 200 quickly; a worker applies it through the state machine. Ordering problems are handled by the conditional transitions.

6. Payouts

Payouts are a batch job: once a day, compute each seller's available balance (charges past the refund/dispute hold window, minus fees and refunds), create a payout record, write ledger entries moving money to "payout in transit", and call the bank transfer API with an idempotency key. The bank's confirmation (often a day later) settles it.

7. Durability and availability

  • Postgres with synchronous replication for the payments and ledger databases; money data favors consistency.
  • If a processor is down, fail over to a second processor for new payments; never retry a payment on a different processor without first confirming the first one didn't charge.

What's expected at each level

  • Mid-level: a payment flow through an external processor, a status field, and awareness that retries must not double-charge.
  • Senior: idempotency keys at both hops, a state machine with conditional transitions, webhook handling, a double-entry ledger.
  • Staff: reconciliation as a first-class component, unknown-outcome handling, payout holds and chargebacks, processor failover, and keeping PCI scope minimal.