Low-level designCore8 min
Producer-consumer
Producers put work on a queue and consumers take it off, so each side runs at its own pace. Give the queue a limit, and a fast producer is slowed to what the consumers can handle instead of filling memory: that's backpressure.
Uploads in, resized photos out
The uploader puts photos on a queue; the resizer takes them off. Neither calls the other.
1 · The situation
Uploads in, resized photos out
A photo-sharing app accepts uploads and makes a small version of each photo for the feed. Uploads arrive in bursts, and resizing is slow, so the two run separately: an Uploader puts each new photo on a PhotoQueue, and a Resizer takes photos off and resizes them. This shape is producer-consumer: the uploader produces work, the resizer consumes it, and the queue between them absorbs the difference in pace.
The first two programs are simulations in whole seconds, not threads: each second the uploader offers 10 photos and the resizer handles 4. Rates are what this part is about, and a simulation shows them exactly.
class PhotoQueue {
private final Deque<Integer> photos = new ArrayDeque<>();
boolean offer(int photo) {
photos.addLast(photo);
return true;
}
boolean poll() {
return photos.pollFirst() != null;
}
int size() {
return photos.size();
}
}The producer and the consumer never call each other; both work through the queue.
2 · The failure
A queue that only grows
Second 1: 10 accepted, 6 waiting Second 2: 10 accepted, 12 waiting Second 3: 10 accepted, 18 waiting Second 4: 10 accepted, 24 waiting Second 5: 10 accepted, 30 waiting
Ten in, four out, every second: six more photos waiting each time.
Photos waiting. The queue grows by 6 a second for as long as the burst lasts.
A queue smooths out a short burst. When the producer is faster for long, an unlimited queue only hides the problem while it gets worse:
- Memory. At 6 photos a second, an hour leaves 21,600 photos waiting in memory, and the process eventually runs out.
- Latency. Every new photo waits behind all the earlier ones, so the feed falls further behind each second.
- No signal. The uploader is told "accepted" every time, so nothing upstream learns that it should slow down.
3 · The fix
Give the queue a limit
Start from the first version and cap the queue. A full queue refuses more work, and the producer must wait. That pushes the slowness back to where the work comes from: backpressure.
PhotoQueue takes a capacity, here 8, and offer() refuses a photo when it's full. In the simulation the uploader stops for the rest of the second; with real threads it would wait.
class PhotoQueue {
private final Deque<Integer> photos = new ArrayDeque<>();
private final int capacity;
PhotoQueue(int capacity) {
this.capacity = capacity;
}
boolean offer(int photo) {
if (photos.size() == capacity) return false;
photos.addLast(photo);
return true;
}
boolean poll() {
return photos.pollFirst() != null;
}
int size() {
return photos.size();
}
}Second 1: 8 accepted, 4 waiting Second 2: 4 accepted, 4 waiting Second 3: 4 accepted, 4 waiting Second 4: 4 accepted, 4 waiting Second 5: 4 accepted, 4 waiting
After the first second fills the queue, the uploader is held to the resizer's pace: four a second, never more than eight waiting.
With real threads, one uploader and two resizers handle 40 photos through a queue of four:
Photos resized: 40 of 40 Left in the queue: 0
4 · The idea
A buffer, and a policy for when it's full
In producer-consumer, producers and consumers share only a queue. Neither knows how many of the other there are, and either side can run more threads without the other changing. A blocking queue adds the waiting: put() blocks while it's full, take() blocks while it's empty, and all the locking is inside it.
A full queue makes the producer wait until a consumer frees a place.
When the queue is full | What happens | Fits |
|---|---|---|
Block the producer | It waits for space | Producers inside your program, such as a file reader |
Reject the new item | The producer gets an error, such as HTTP 429, and may retry later | Requests from outside that can be retried |
Drop the oldest or newest | Some work is lost on purpose | Sensor readings or metrics where the latest matters most |
Grow without limit | Nothing, until memory runs out | Only bursts known to be short |
5 · What it costs
Choosing the size
The capacity is a trade-off:
- Too small, and producers stall on every short burst the queue could have absorbed.
- Too large, and the queue hides overload for a long time, with items waiting minutes before anyone works on them.
- Blocking can deadlock. A consumer that must put something back into a full queue it's also emptying, or a producer holding a lock a consumer needs, can wait forever.
A good size covers the longest burst you expect at the consumers' rate, and no more. If the queue is full most of the time, the queue isn't the problem: the consumers are too slow, and the fix is more of them or less work per item.
6 · Getting it right
Start, stop, and watch the length
- Plan the stop. A blocking queue has no "closed" state of its own, so producers commonly send an end marker per consumer, as here. Python 3.13 added
Queue.shutdown()for the same job. - Never block while holding a lock that the other side needs to make progress.
- Watch the queue's length. A queue that keeps growing, or keeps hitting its limit, is the earliest sign that consumers can't keep up.
- Hand over whole items. Once a producer puts a photo on the queue, it shouldn't touch it again; the consumer owns it now.
Language | A bounded blocking queue |
|---|---|
Java |
|
C++ | None in the standard library; a deque with a mutex and two condition variables, as here |
JavaScript | None built in; async code awaits space, and Node's streams apply backpressure between readable and writable ends |
Python |
|
7 · Look-alikes
Queues at every scale
The same shape appears inside one program and between whole systems.
Idea | How it relates |
|---|---|
Producer-consumer | Threads in one program sharing a bounded queue |
Thread pool | A queue of tasks with a fixed set of consumer threads, covered next |
Pipeline | Several producer-consumer stages in a row, each stage's consumers producing for the next |
Message queue | The same idea between programs, through a broker that stores messages |
Observer | Pushes every event to every listener at once; there's no queue and no backpressure |
8 · In practice
Backpressure in the libraries
Java's BlockingQueue documentation describes the interface as designed mainly for producer-consumer queues, and suggests end-of-stream or poison objects for shutting it down. Node.js streams apply the same idea between a fast readable source and a slow writable destination: when the destination's buffer is full, write() returns false, and the source should pause until a 'drain' event.
Whenever data flows from something fast to something slow, ask the question this page answers: what happens when the slow side falls behind?
Check yourself
4 questions
Takeaways
Remember this
- Producers and consumers share only a queue, so each side runs at its own pace and scales on its own.
- An unbounded queue turns sustained overload into growing memory and latency, with no signal upstream.
- A bounded queue applies backpressure: when it's full the producer waits, or is refused, and is held to the consumers' pace.
- Size the queue for the bursts you expect; if it's always full, add consumers or reduce work per item.
- Plan the shutdown: one end marker per consumer, or a queue that supports closing.