System designDeep dive13 min

Event sourcing & CQRS

Most databases keep the latest answer and throw away how it was reached. Event sourcing keeps every change as a fact and computes the answer from them; CQRS builds separate, fast views for reading. Together they buy a full history at the price of a very different way of working.

1 · The problem

A row remembers the answer and forgets the story

A warehouse tracks one product, the K-200 kettle. Its row in the inventory table says stock = 45. Every sale, return and broken box has updated that one number in place. Then the questions come:

Purchasing: how many kettles from the supplier Brightway arrived damaged this year?
Accounting: what was the stock at midnight on September 30?
Support: the shelf holds 44 kettles, the system says 45; where did the difference come from?

The row can't answer any of them. Each update overwrote the value before it, and the reasons were never stored.

Event sourcing stores something else. Instead of the current value, it keeps every change as an event: a fact, in the past tense, with what happened and why. The current stock is not stored at all; it is computed by adding the events up:

Version	Event	Stock after
1	StockReceived: 50 from Brightway	50
2	ItemsSold: 3, order 7121	47
3	ItemDamaged: 1, dropped in aisle 4	46
4	ItemsReturned: 1, order 7121	47
5	ItemsSold: 2, order 7188	45

Both designs say 45. Only the second can say why: the damaged kettle, the return, the two sales, each with its order number and time. That history is the point of the pattern, and almost everything else on this page is either a way to make it practical or a price paid for it.

2 · The idea

Store what happened, and compute what is

Each thing whose history matters gets its own stream: an ordered list of events, here kettle-K200. Each new event is appended at the end with the next version number. To load the kettle, the service reads its stream from the start and applies the events in order, a step called rehydration:

Current stock is a function of the events

javascript

function stockOf(events) {
  let stock = 0;
  for (const e of events) {
    if (e.type === 'StockReceived') stock += e.qty;
    if (e.type === 'ItemsSold') stock -= e.qty;
    if (e.type === 'ItemDamaged') stock -= e.qty;
    if (e.type === 'ItemsReturned') stock += e.qty;
  }
  return stock;
}
// stockOf(kettleK200) → 45

Three rules make the log trustworthy:

Events are never changed or deleted. The store only appends.
Mistakes are corrected by new events. If a count was wrong, the fix is StockAdjusted: −1, recount found 44, not an edit to an old event. The mistake and its correction both stay visible.
Events record intent, not just values. "ItemDamaged: 1, dropped in aisle 4" is useful forever. "Stock changed to 46" is just the old row again, written down more often.

The event store that holds the streams can be a database built for it or an append-only table in an ordinary database. Either way it must read one stream quickly and reject conflicting appends (next section). A log-based broker like Kafka is good at sending events on to other services, but it lacks both features, so it isn't a substitute for an event store. And these events are the service's own private record. When other services need to know, the service usually publishes separate, stable events for them (Event-driven architecture).

3 · The conflict

Two writers on one stream: the version number decides

Weeks later the kettle's stream is at version 41 and the stock is 2. Two orders arrive at the same moment, each for 2 kettles. Two instances of the order service each load the stream, see 2 in stock, and approve their order. Without protection, both append ItemsSold: 2, and the stock becomes −2.

Event stores prevent this with optimistic concurrency. Each append says which version the writer read, and the store accepts it only if the stream is still at that version:

Order A

Event store

Order B

read: v41, stock 2

append Sold 2, expect v41

ok: now v42

append Sold 2, expect v41

rejected: stream is at v42

reload: v42, stock 0

refuse order: sold out

Both orders read version 41, but only one append can follow it. The loser reloads, sees the new event, and decides again with the true stock.

No lock is held while the order is being checked; conflicts are detected only at the append, which is why the approach is called optimistic. It is the same idea as compare and set on a database row (Transactions & isolation levels). The version check protects one stream at a time. A rule that spans two streams, such as "never sell more kettles than the warehouse holds across all its sites", still needs a design of its own.

4 · The reads

Queries read separate views built from the events

Loading one kettle is easy: read its stream. Now the shop wants "all products with fewer than 10 in stock". An event store is built to read one stream by its ID; answering that means replaying every product's history for every query. Event sourcing on its own is very bad at queries.

The answer is CQRS, command query responsibility segregation: use one model to change data and different models to read it. The write side handles commands, checks the rules and appends events. A projector reads every new event and updates read models, each a table shaped for exactly one kind of question:

One set of events, three read models. Each is an ordinary table that answers its question with a simple lookup, and none of them is the source of truth.

The two sides can use different databases, different shapes and different numbers of machines. The read side often gets far more traffic, so it can scale on its own. CQRS doesn't require event sourcing: a service with an ordinary database can publish its changes through an outbox and build read models the same way (Distributed transactions: 2PC & sagas). Event sourcing, on the other hand, almost always needs CQRS, because without read models it can barely be queried.

The price is lag. The projector applies an event a moment after it is appended, so a read model can be a little behind:

Order service

Event store

Projector

Shop stock

append Sold 2 (v42)

still shows 2

event v42

stock 2 → 0, at v42

Between the append and the projection, the shop page shows 2 kettles that are already sold. The write side still rejects any order that would oversell.

Two habits keep the lag harmless. The write side, not the read model, makes the decisions: an order is checked against the stream, so a stale shop page can mislead a customer but can't oversell. And each read model stores the last event version it applied, as above, so a redelivered event is recognized and skipped (Delivery semantics).

5 · The payoff

Replaying the log answers questions nobody asked at the time

Back to the three questions the row couldn't answer. With the events kept, each one becomes a replay:

Damaged items from Brightway this year: write a new projector that counts ItemDamaged events by supplier and month, and run it over the whole history. The new read model is complete on its first day, even though nobody planned for it.
Stock at midnight on September 30: replay the kettle's events up to that time and stop. This is a temporal query.
Shelf says 44, system says 45: read the stream. Every change is there with its reason, and the missing kettle can be traced, then fixed with a StockAdjusted event.

Replay also repairs bugs in read models. If the damage report counted returns as damage for a month, fix the projector, rebuild the read model from the first event, and switch to the rebuilt table. The events were right all along; only a view of them was wrong.

There is one danger. A replay runs your code over old events again, and some code does more than update tables:

During a replay	What goes wrong	The usual answer
A handler emails Brightway when stock drops below 10	Brightway gets a year of old alerts again	Side effects run only for live events, never during replay
A handler looks up today's price for each sale	Old sales get today's price	Store the looked-up value in the event when it happens

The rule behind both: a replay must depend only on what is in the events. Anything learned from or done to the outside world at the time belongs in the event itself, or it can't be reproduced.

6 · The price

Keeping everything forever creates problems a table never has

A busy product collects thousands of events. Loading it means reading all of them on every command. The usual fix is a snapshot: every N events, save the computed state with the version it covers. Loading then reads the latest snapshot and only the events after it. With a snapshot every 500 events:

No snapshots
Snapshot every 500

Without snapshots, loading reads the whole stream: 2,000 events for a 2,000-event stream. With a snapshot every 500 events, it reads one snapshot and at most 499 events. Snapshots are only a cache; they can be thrown away and rebuilt from the stream.

Snapshots fix speed. The other costs come from the same source, that nothing is ever deleted or rewritten:

Problem	Why it is hard here	The usual answer
Event shapes change	Years of old events keep their old shape	Readers that ignore new fields, a version in each event, and upcasting (old events converted as they are read)
A bug wrote wrong events	Fixing the code doesn't fix the stored events	Compensating events that correct the effect
Personal data must be erased	Events can't be deleted without breaking the stream	Keep personal data outside the events and refer to it by ID
Read models lag	Projections are built after the append	Decide on the write side; show users their own changes from it
Unfamiliar to most teams	Testing, debugging and operations all change	Use it only where the history pays for itself

Erasure needs a word more. When personal data can't be kept out of events, some teams encrypt it with one key per person and delete the key on request, a technique called crypto-shredding. Whether destroying the key legally counts as erasing the data is debated, so keeping personal data out of the events in the first place is the safer design. Rewriting stored events to change their shape or remove data is possible, but it breaks the audit trail the pattern exists for, and is a last resort.

7 · In practice

Use it where the history is worth more than the simplicity

Event sourcing is not a choice for a whole system. It is a choice for one kind of data, and most data doesn't need it. Moving to it, or away from it, later is expensive, so the choice deserves care:

Data	History matters?	Use
Stock movements, payments, ledgers	Yes: audits, disputes and "why" questions	Event sourcing with CQRS read models
Product catalog shown to millions	No, but reads vastly outnumber writes	CQRS alone: an ordinary database and read models
User profile, settings	No	An ordinary table
A prototype or short-lived tool	Not yet	An ordinary table

Signs it fits: the business already talks in events ("received", "shipped", "returned"); questions about the past keep coming up; many writers update the same things; new reports keep being asked of old data. Signs it doesn't: the data is simple create-read-update-delete, every screen must show the latest state instantly, or nobody on the team has run an event-driven system before.

If you adopt it:

One stream per thing whose history matters, appended with an expected version.
Events in the past tense that carry intent, including any outside value they depended on.
Never edit events; correct with new ones, and convert old shapes as they are read.
Read models for every query, each storing the last version it applied, rebuilt by replay when they are wrong.
Snapshots when streams get long, treated as a disposable cache.
Personal data outside the events, referred to by ID.

Check yourself

5 questions

1. The kettle's stock was miscounted: the system says 45, the shelf holds 44. What is the event-sourced way to fix it?

2. Two order instances both read the kettle stream at version 41 with 2 in stock, and both try to append "ItemsSold: 2" with expected version 41. What happens?

3. Why does an event-sourced service almost always need CQRS read models?

4. Purchasing asks for damaged kettles per supplier for the past year, a report nobody planned. What does event sourcing let you do?

5. A projector that emails the supplier when stock drops below 10 is rebuilt by replaying a year of events. What must the design ensure?

Takeaways

Remember this

Event sourcing stores every change as an immutable event in a stream and computes the current state by replaying them.
Mistakes are corrected with new events, never edits, so the history stays a true audit trail.
Appends carry an expected version, so two writers on one stream can't both succeed on stale state.
CQRS separates the write side from read models built by projectors; reads are fast but lag behind the events.
Replay builds new reports from all of history, but long streams need snapshots, changing event shapes need upcasting, and personal data belongs outside the events.