Chat delivery & sync

A messaging app for amateur sports clubs has one-to-one chats and club groups of up to 500 members. Most people use it on a phone and a laptop. Mira, a team captain, posts "Training moved to 7pm" to the club's group of 120. Mira's phone is on a train and drops the connection just after sending; some members are online, some have the app closed, and one member's laptop has been shut since Friday.

Every one of them must end up with the same conversation. That splits into requirements that pull against each other:

Read the right-hand column as the design. The rest of the page builds each piece and shows how they fit, from the moment Mira presses send to the moment the closed laptop opens on Monday.

Mira's phone sends the message, the train enters a tunnel, and no acknowledgment comes back. Did the server get it? The phone can't know, so it retries, and without care the group sees the message twice. The fix is the same as for any retried request (Idempotency): the client creates an id for the message before the first attempt and reuses it on every retry, and the server stores the id with the message and ignores repeats.

The open Matrix chat protocol builds this into its API: a client sends each message with a transaction id in the request path, which exists so the server can tell a new request from a retransmission and make the request idempotent. The server, not the client, then decides where the message goes in the conversation:

The order of a conversation is the order the server appended messages, not the times on senders' phones. Phone clocks disagree and can be changed, and two messages sent in the same second need a winner (Time, clocks & ordering). Each conversation gets a sequence that only grows, and every device shows messages in that order. Until the acknowledgment arrives, the sender's app shows the message as pending in its own position, then moves it if the server placed it differently (Offline-first & optimistic UI).

Nearly every read in a chat app is "messages in this conversation, near this point": the latest page, a page before it, the context around a search hit. So messages are stored together by conversation, ordered by their sequence, in a log that is only appended to. Discord stores its messages this way:

Bucketing has a cost: a quiet channel's last 50 messages may be spread over many buckets, so loading them means querying partition after partition. Discord's fix was to track empty buckets per channel and skip them.

The opposite problem is a channel everyone reads at once, such as a huge community during a big event. That partition becomes hot (Hot partitions & skew). Discord put services between its API and the database that perform request coalescing: if many users request the same row at the same time, the database is queried once. Requests are routed by consistent hashing on the channel, so concurrent requests for one channel reach the same service instance and can be merged (Sharding & partitioning).

Members with the app open hold a long-lived connection to the service, usually a WebSocket (WebSockets & long polling). The message has to get from the server that accepted it to every connection belonging to a member of the group. Slack does this with two kinds of stateful server:

In Slack's design, channel servers hold recent channel history in memory and are mapped to channels by consistent hashing; a failed one is replaced in under 20 seconds. Gateway servers hold users' WebSocket subscriptions and run in several regions, so clients connect nearby. Slack reports delivering messages across the world within 500 ms. Typing indicators take the same path but are never stored.

Live push is a fast path, not the guarantee. A connection can drop between the server sending and the device receiving, and a device with the app closed has no connection at all; it gets a push notification as a hint (Presence & push notifications). What makes delivery complete is the next step: every device catches up from storage when it reconnects.

On Monday the closed laptop opens. Rather than asking "what's new?", each device keeps a cursor: an opaque token marking the last point it has. The Matrix protocol shows the pattern. A client's first sync returns recent messages and a next_batch token; each later sync sends that token back as since and gets only what came after, with the server holding the request open until something arrives.

A device that was away a long time can't be sent everything at once. Matrix's answer is a limited timeline: if too many events arrived since the cursor, the server returns only the most recent ones plus a summary of state changes, and the client may have gaps, which it fills from a separate history API when the user scrolls back:

The cursor belongs to the device, not the person, because each device has seen a different amount. Mira's phone, which never went offline for long, syncs a handful of messages; the laptop gets a limited timeline. Both end up showing the same conversation in the same order, because both read the same log.

"Delivered" and "read" are events too. Matrix sends read receipts as events in the room, each recording which user read which message, and when. Reading a message on the laptop should clear the unread badge on the phone, so a receipt is synced to the reader's own devices as well as shown to the sender. For a group of 120, receipts are summarized ("read by 87"), because a receipt per member per message would outnumber the messages.

End-to-end encryption changes the picture, because the server can no longer read messages. WhatsApp's multi-device design shows the consequences:

• Each device has its own identity key, rather than one per person.
• The sending client fans out: it encrypts and sends the message once for each device in the recipient's and the sender's own device lists, each with its own pairwise session.
• Messages aren't stored on the server after delivery, so a new device can't download history from the server.
• Groups use Sender Keys from the Signal Protocol, so a sender doesn't encrypt separately for every member's device.

Encrypted sessions make this harder: a message from Alice to Bob may need sessions to all of Bob's devices, and to Alice's other devices so they get a copy, and devices come and go. With end-to-end encryption, server-side search and server-held history also give way, and those features move to the devices.

Mira's group has 120 members. If each has 2.4 devices on average, one message means 120 × 2.4 = 288 device deliveries. Sent one at a time from the server that accepted it, that is 288 sends for every message. Fanning out in two stages, as in the flow above, changes who does the work:

The total number of pushes doesn't shrink, but it spreads across gateways that already hold the connections, and the channel server's work grows with the number of gateways, not members. Stored once in the conversation log, the message is read by each device's sync rather than copied into hundreds of inboxes. Copying each message into every member's inbox is the fan-out-on-write approach, weighed in Fan-out on write vs read; this design keeps one log per conversation.