System designDeep dive8 min

Time-series databases

Metrics, sensor readings and prices arrive as an endless stream of timestamped numbers, are read back as ranges and averages, and lose value as they age. Time-series databases are built around that shape: they cut data into time blocks, compress neighboring points to a byte or two, and keep old data as summaries.

1 · The workload

Time-series data arrives in time order and is read back in ranges

A solar monitoring company watches 400,000 home solar installations. Every 30 seconds each inverter reports six numbers: power, voltage, current, temperature, daily energy and status. That's 400,000 × 6 ÷ 30 = 80,000 points a second, or about 6.9 billion a day. Owners open an app to see today's curve; the company's engineers chart a whole region's output for the last year.

The workload looks nothing like an online store's orders:

Property	Time-series data
Writes	Almost all appends of new points; old points are rarely changed
Reads	One or many series over a time range, usually summarized: averages, maximums, rates
Recency	Recent data is read far more; in one large monitoring system at least 85% of queries covered the last 26 hours
Deletion	Whole time ranges expire at once, by age

A general-purpose database can store this, but it pays for features the workload doesn't need: a row and an index entry for every point, updates in place, deletion row by row. At 80,000 points a second that overhead decides the size of the bill.

2 · Series

A series is a name and labels, followed by a list of points

Most time-series databases model data as series. A series is identified by a metric name and a set of labels, and holds a list of (timestamp, value) points:

One series and its points

inverter_power_watts{site="s-104233", region="bavaria"}
  2026-06-01T12:00:00Z  4120
  2026-06-01T12:00:30Z  4135
  2026-06-01T12:01:00Z  4135
  2026-06-01T12:01:29Z  4098

The database keeps an index from metric names and labels to series, so "power for every site in Bavaria" finds the right series without scanning the rest. The number of series is what drives memory and index size, and every distinct label combination is a new one (Observability: metrics, logs & traces):

400,000 sites × 6 metrics = 2.4 million series.
Add a label for each of a site's 20 panels and it becomes 48 million.
Add a label with a request or user ID and there's no limit at all.

So labels describe where a number came from, and anything unbounded belongs in logs or in a regular database.

3 · Time blocks

Data is cut into time blocks, so writes stay small and old data drops whole

Prometheus, a widely used metrics database, shows the standard layout. New points go into a head block kept in memory and protected by a write-ahead log (Storage engines: B-trees, LSM trees & WAL). Every two hours the head is written out as an immutable block with its own chunks of points and its own index. Later, background compaction merges blocks into bigger ones, up to 10% of the retention period or 31 days:

All 80,000 points a second are appended to the head; nothing on disk is rewritten in place. A query for the last hour reads the head, and a query for last week opens only the blocks that overlap that week.

Cutting by time pays off three times:

Queries skip most data. Each block covers a known time range, so a query opens only the blocks it needs.
Recent data is small and hot. The head and the newest blocks fit in memory, where most queries land.
Expiry is free. Retention deletes whole blocks once all their data is past the limit. TimescaleDB, which adds time series to PostgreSQL, does the same with time-range chunks: dropping a chunk deletes a file, while DELETE row by row is slow and leaves space to clean up.

4 · Compression

Neighboring points look alike, so they compress to a byte or two

A raw point is a 64-bit timestamp and a 64-bit float: 16 bytes. Consecutive points in a series barely differ: Facebook's Gorilla, an in-memory time-series database, stores them in about 1.37 bytes each, a 12× reduction, with two tricks.

Timestamps: store the change in the gap. Points arrive at a steady interval with small jitter, so instead of the timestamp, store how much the gap changed from the previous gap:

Timestamp	Gap	Change in gap
12:00:00	(first point, stored in full)	–
12:00:30	30 s	–
12:01:00	30 s	0
12:01:29	29 s	−1
12:02:00	31 s	+2

A change of 0 is stored as a single bit, and in Gorilla's data about 96% of timestamps compressed to that one bit. Values: XOR with the previous value. Nearby values share their sign, exponent and leading bits, so the XOR of the two is mostly zeros and only the bits that differ are stored. An identical value, like the repeated 4135 above, takes one bit; that was about 51% of Gorilla's values.

Raw, 16 bytes a point: 110.6 GB a day
Compressed, 1.37 bytes: 9.5 GB a day

6.9 billion points a day for the solar company. Prometheus documents 1 to 2 bytes per sample, and its disk sizing rule is retention × samples per second × bytes per sample.

Compression works per series and per block: points are encoded in order, one after another, and a query decodes a whole chunk to read any point in it. That suits time ranges and summaries, and is a poor fit for random single-point lookups or edits to old data.

5 · Rollups

Old data is kept as summaries, not as points

Nobody needs a year of power output at 30-second resolution: a chart can't show a million points per line anyway. The fix is downsampling, also called rollups: store summaries of fixed intervals, such as each 5 minutes or each hour, and query those for long ranges. For one series over a year:

Raw, every 30 s: 1,051,200 points
5-minute summaries: 105,120 points
1-hour summaries: 8,760 points

365 × 24 × 120 raw points, against 365 × 24 × 12 and 365 × 24. A regional chart over 50,000 sites multiplies each bar by 50,000.

A summary must keep enough to answer later questions correctly. An average of averages is wrong when intervals have different counts, so Thanos, which stores Prometheus data long-term, keeps the count, sum, minimum and maximum for each interval; the average is sum ÷ count. It creates 5-minute summaries for blocks older than 40 hours and 1-hour summaries after 10 days.

Thanos is explicit that downsampling is about query speed, not disk space: it adds summary blocks next to the raw ones. Space is saved by giving each resolution its own retention. TimescaleDB combines continuous aggregates, summaries it keeps up to date as data arrives, with retention policies that drop the raw chunks, so history survives as summaries.

6 · In practice

Plan retention per resolution, and keep series counts bounded

The solar company's plan, sized with the 1.37-byte figure for raw points:

Resolution	Kept for	Used by
Raw, 30 s	30 days (about 285 GB)	Today's curve in the app, fault diagnosis
5-minute summaries	2 years	Monthly reports, comparing sites
1-hour summaries	Forever	Year-over-year output, regional planning

The usual choices sort by what you already run and how you query:

Prometheus for infrastructure metrics: it pulls them from each service over HTTP, keeps 15 days by default, and pairs with Thanos or a similar system for long-term storage.
TimescaleDB when the data must be joined with regular tables in SQL, such as readings with customer accounts.
A columnar analytics database when the questions are ad hoc analysis over many dimensions (OLTP vs OLAP & columnar storage).
A plain table with a time index, partitioned by month, when the volume is modest and the team would rather not run another database.

Count the series first
Sites × metrics × label values; reject labels with unbounded values before they ship.
Size from samples per second
Retention × samples per second × bytes per sample, for each resolution.
Roll up early
Keep count, sum, min and max per interval, and point long-range charts at the summaries.
Expire by time
Retention that drops whole blocks or chunks, never row-by-row deletes.
Plan for late data
Devices that were offline send old points; decide how far back the database will accept them.

Check yourself

4 questions

1. A time-series database cuts data into 2-hour blocks. Which is NOT a benefit?

2. Points arrive every 30 seconds with gaps of 30, 30, 29 and 31 seconds. What does delta-of-delta encoding store for the last three gaps?

3. An engineer adds a panel_id label (20 panels per site) to a metric collected from 400,000 sites. What happens to that metric's series count?

4. Why do downsampled summaries keep count, sum, min and max instead of just the average?

Takeaways

Remember this

Time-series data is appended in time order, read back as ranges and summaries, and expires by age; general-purpose databases pay for features it doesn't need.
A series is a metric name plus labels, and every distinct label combination is a new series, so series counts must stay bounded.
Time blocks keep writes in a small in-memory head, let queries skip unrelated ranges, and let retention drop whole blocks.
Delta-of-delta timestamps and XOR-encoded values shrink a 16-byte point to about 1.37 bytes.
Long-range queries read rollups that keep count, sum, min and max, and each resolution gets its own retention.