Lesson 12 of 15 Case Studies

Design a Chat System (WhatsApp)

Chat system design like WhatsApp: WebSocket gateways, message routing, ordering, storage, delivery receipts, presence, group chats and offline push.

50-60 min advanced 0/15 lessons done
Course contents Lesson 12 of 15 · 0 done
On this page (13 sections)
  1. The problem
  2. 1. Requirements
  3. 2. Connections
  4. 3. Sending a message
  5. 4. Ordering
  6. 5. Storage
  7. 6. Group chats
  8. 7. Presence
  9. 8. Multi-device and sync
  10. 9. Deep dives
  11. Key takeaways
  12. Exercises
  13. Quiz

The problem

Design a messaging service like WhatsApp or Messenger: one-to-one and group chats, delivered in real time, with history on every device.

1. Requirements

Functional: 1:1 and group messages (groups up to ~500), sent/delivered/read receipts, online presence, message history, push notifications when offline.

Non-functional: 500M DAU; low latency (deliver in under ~200 ms when both users are online); messages never lost and shown in order within a conversation; history kept indefinitely.

Estimates: 500M users × 40 messages/day = 20B messages/day ≈ 230k/s average, ~700k/s peak. At ~100 bytes each that’s ~2 TB a day of message data (more with metadata and replicas). Concurrent connections: maybe 100M+ at peak.

2. Connections

Clients need server push, so each online client holds a WebSocket to a chat gateway server (lesson 9). One gateway can hold on the order of 100k–1M idle connections, so we need hundreds of gateways. Gateways are stateful: each knows which users are connected to it. A session registry (Redis) maps user_id -> gateway_id.

 phone A ==ws==> gateway 7 ---+                          +--> gateway 3 ==ws==> phone B
                              |                          |
                              v                          |
                      chat service ---> message store    |
                         |   ^  (Cassandra, by conversation)
                         |   +-- session registry (user -> gateway)
                         +--> push service (APNs / FCM) for offline users

3. Sending a message

  1. A sends {client_msg_id, conversation_id, text} over its WebSocket to gateway 7.
  2. The chat service assigns a per-conversation sequence number, persists the message, then acknowledges to A (“sent”, one tick). client_msg_id makes retries idempotent: a resend after a lost ack doesn’t create a duplicate.
  3. It looks up B in the session registry. If B is online on gateway 3, it forwards the message there, gateway 3 pushes it down B’s socket, and B’s client acks (“delivered”, two ticks).
  4. If B is offline, it sends a push notification; B fetches missed messages when it reconnects.

Gateways can talk to the chat service through a message bus partitioned by conversation, which also gives retries and ordering.

4. Ordering

Clocks on phones and servers disagree, so don’t order by timestamp alone. A per-conversation monotonic sequence number (assigned by the single partition/owner for that conversation) gives a total order inside each chat, which is all users need. Clients sort by it and detect gaps (“I have 41 and 43, fetch 42”).

5. Storage

Access pattern: “append a message to a conversation” and “read the latest N messages of a conversation”. That’s a perfect fit for a wide-column store (Cassandra/HBase/ScyllaDB) with:

partition key:  conversation_id
clustering key: seq (descending)
columns:        sender_id, body, sent_at, type

A conversation’s history lives in one partition, sorted. Very large group histories can be bucketed by time (conversation_id + month). Media (photos, video) goes to object storage; messages carry a link.

Per-user state (last read seq per conversation, unread counts) lives in a separate table keyed by user.

6. Group chats

For a message to a group of 200: store it once in the group’s partition, then fan out delivery to each member’s gateway (or push). For small groups this fan-out is cheap. For very large broadcast channels, members pull on open instead of receiving a push per message (the same push/pull trade-off as in the news feed).

7. Presence

Online status comes from heartbeats on the WebSocket. The gateway updates last_seen in a presence store; a user is “online” if a heartbeat arrived in the last ~30 s. Presence changes are fanned out only to contacts who currently have the chat open, and are batched or throttled, because presence chatter can easily exceed message traffic.

8. Multi-device and sync

Each device tracks the last seq it has per conversation. On reconnect it asks “give me everything after seq X” for each active conversation, so history is consistent on all devices.

9. Deep dives

  • Gateway failure: clients reconnect (with jittered backoff) to another gateway; the registry is updated; unacked messages are redelivered from storage.
  • End-to-end encryption: clients encrypt (Signal protocol); servers route opaque ciphertext, which also means server-side search is not possible.
  • Hot groups: rate-limit senders and shard delivery work across workers.

Key takeaways

  • Online clients keep WebSockets to stateful gateways; a registry maps users to gateways.
  • Persist first, then acknowledge, then deliver; client message ids make retries idempotent.
  • Order with per-conversation sequence numbers, not timestamps.
  • Store messages partitioned by conversation, sorted by seq; fan out group messages on delivery; throttle presence.

Exercises

  1. Write a Python function that, given a list of received seq numbers for a conversation, returns the missing ranges to fetch.
  2. Design read receipts for groups (“read by 37 of 200”). What do you store and when do you update it?
  3. Estimate how many gateway servers you need for 100M concurrent connections at 500k connections per server, with 30% headroom.

Quiz

  1. Why are chat gateways stateful, and how does the chat service find the right one?
  2. Why not order messages by the sender’s timestamp?
  3. Why is conversation_id a good partition key for messages?
Show answers
  1. Each holds open WebSocket connections for specific users; a session registry maps user_id to the gateway holding their connection.
  2. Device clocks drift and can be wrong or manipulated, so timestamps can produce inconsistent orders; a per-conversation sequence number is authoritative.
  3. The main query reads the latest messages of one conversation; keeping them in one sorted partition makes that a single, efficient read.

Finished this lesson?

Mark it complete to track your progress (saved in this browser).