Courses › System Design › Patterns

Generating Unique IDs

Auto-increment, UUIDv4 vs v7, Snowflake IDs and range allocation — and public vs internal IDs.

Every design needs IDs: for users, posts, orders, short links, messages. On one database an auto-increment column is fine. Once you have many database shards or many services creating records in parallel, you need IDs that are unique without coordination, and often sortable by time too.

What do you need from the ID?

Ask these before picking a scheme — the answers decide it:

  • Uniqueness across machines — always required.
  • Roughly time-ordered? Lets you sort and paginate by ID ("newest first") without a separate timestamp index. Feeds and chat need this.
  • Size? 64-bit integers are compact and index well; 128-bit values (UUIDs) are twice the size in every index that holds them.
  • Guessable? Sequential IDs leak business volume (order #10452 tells a competitor your sales) and invite enumeration. Public IDs should usually be opaque.
  • Short? For URLs people type or share, length matters (see Design a URL Shortener).

The options

1. Database auto-increment

Simple and compact — but a single writer. With shards, you can give each shard its own offset (shard 1: 1, 4, 7…, shard 2: 2, 5, 8…), which works until you need to add a shard.

2. UUIDs

128 random bits (UUIDv4), generated anywhere with no coordination and a negligible chance of collision. The cost: they're large, and random keys are bad for B-tree indexes — every insert lands on a random page, so the working set of the index stops fitting in memory.

UUIDv7 fixes the ordering problem: the first 48 bits are a millisecond timestamp and the rest are random. You keep "generate anywhere", and inserts append to the end of the index like a sequence.

3. Snowflake-style IDs

Twitter's Snowflake packs a 64-bit integer from three parts:

| 1 unused | 41 bits: ms since custom epoch | 10 bits: machine ID | 12 bits: sequence |
             ~69 years of timestamps          1,024 generators      4,096 IDs per ms per machine
  • Time-sortable, because the timestamp is in the high bits.
  • No coordination per ID: each generator only needs a unique machine ID, assigned once at startup (from ZooKeeper, etcd, or a config table).
  • 64 bits, so they fit in a BIGINT.

Many companies use variants. Instagram, for example, generated IDs inside each Postgres shard with a 41-bit timestamp, a 13-bit shard ID and a 10-bit sequence — so the shard is encoded in the ID itself and lookups need no directory.

[!WARNING] Clocks go backwards If NTP moves a generator's clock back, it could reissue IDs it already used. Snowflake generators remember the last timestamp and refuse to generate (or wait) until the clock passes it again. Mention this — interviewers often ask.

4. Ticket server / range allocation

A central service hands out blocks of IDs (say 1,000 at a time) from a database counter. Each app server uses its block from memory and asks for another when it runs out. One database write per 1,000 IDs, compact sequential numbers, and the ticket server is only on the critical path when a block runs out — run two of them (one hands out odd blocks, one even) for availability.

Comparing them

Public vs internal IDs

You don't have to show users the same ID you index on. A common pattern:

  • Internal: a Snowflake or sequential BIGINT primary key, compact and fast to join on.
  • External: an opaque string — the ID encoded in base62, a random token, or an encrypted form of the internal ID — so URLs don't reveal counts or allow walking the ID space.
62^7 ≈ 3.5 trillion   → 7 characters covers any realistic number of links
62^11 > 2^64          → any 64-bit ID fits in 11 characters

Ordering is only approximate

Snowflake and UUIDv7 IDs are ordered by each generator's clock, so IDs from different machines created within the same few milliseconds can sort slightly out of creation order. That's fine for feeds and pagination. If you need a strict total order (a chat conversation's message sequence, a ledger), assign a per-partition sequence number from a single writer for that partition instead.

[!INTERVIEW] What a strong answer sounds like "Tweets need IDs that are unique across hundreds of servers and sortable by time, so the timeline can paginate with WHERE id < :cursor. I'll use 64-bit Snowflake IDs: 41 bits of milliseconds, 10 bits of worker ID assigned from ZooKeeper at startup, 12 bits of sequence. That's 4 million IDs per second per worker, more than we need. Generators refuse to issue IDs if the clock goes backwards."