Low-level designCore8 min

Futures and async

A future is a handle to a result that isn't ready yet. Start several slow calls, hold their futures, and wait for all of them together: the total wait becomes the slowest call, not the sum. Then decide what a failure in one of them means for the rest.

Watch it evolve1 / 8

The problem

Three quotes for one trip

TripPlanner asks a flight, a hotel and a car service for quotes and adds them up.

Start

Stage 1

Stage 2

Final

1 · The situation

Three quotes for one trip

A travel site builds a trip from three quotes: a flight, a hotel and a rental car, each from a different remote service. Each call is slow: here 200, 300 and 100 milliseconds, simulated by sleeping. TripPlanner.plan() asks for the three and adds them up. The first version asks one after another.

One call after anotherJava

class TripPlanner {
    private final QuoteService flight;
    private final QuoteService hotel;
    private final QuoteService car;

    TripPlanner(QuoteService flight, QuoteService hotel, QuoteService car) {
        this.flight = flight;
        this.hotel = hotel;
        this.car = car;
    }

    String plan() {
        int f = flight.quote();
        int h = hotel.quote();
        int c = car.quote();
        return "Flight $" + f + ", hotel $" + h + ", car $" + c + ": total $" + (f + h + c);
    }
}

Each program times itself and checks the time against the sum of the three waits, 600 milliseconds; exact times vary from run to run, so it prints only the comparison.

Flight $420, hotel $310, car $95: total $825
Took at least the sum of the waits: yes

2 · The problem

Waiting for each call in turn

The three calls don't depend on each other: the hotel quote doesn't need the flight's price. Yet the planner waits for the flight before it even asks for the hotel. The trip takes as long as all three calls added together, and every service the site adds makes it slower still.

One after another: the sum: 600 ms
All at once: the slowest: 300 ms

The waits are 200, 300 and 100 ms. Done in turn, they add up; done together, the trip waits only for the hotel.

3 · The fix

Start all three, then wait for all three

Start every call first, keep a handle to each result, and only then wait. The handle is a future, or a promise in JavaScript: an object that will hold the result, or the error, once the call finishes.

Java starts each call with CompletableFuture.supplyAsync() and waits with join(). C++ uses std::async, which runs each call on its own thread and returns a std::future. JavaScript passes the three promises to Promise.all(), and Python passes the three coroutines to asyncio.gather().

Java

class TripPlanner {
    private final QuoteService flight;
    private final QuoteService hotel;
    private final QuoteService car;

    TripPlanner(QuoteService flight, QuoteService hotel, QuoteService car) {
        this.flight = flight;
        this.hotel = hotel;
        this.car = car;
    }

    String plan() {
        CompletableFuture<Integer> f = CompletableFuture.supplyAsync(flight::quote);
        CompletableFuture<Integer> h = CompletableFuture.supplyAsync(hotel::quote);
        CompletableFuture<Integer> c = CompletableFuture.supplyAsync(car::quote);
        int fp = f.join();
        int hp = h.join();
        int cp = c.join();
        return "Flight $" + fp + ", hotel $" + hp + ", car $" + cp + ": total $" + (fp + hp + cp);
    }
}

Flight $420, hotel $310, car $95: total $825
Took less than the sum of the waits: yes

The same trip, in about the time of the slowest call.

Flight $420, hotel $310, car unavailable: total $730
Took less than the sum of the waits: yes

One optional call failed; the rest of the plan stands.

The planner holds three services and asks all three at once; what it does with each answer is its own decision.

4 · The idea

Futures, promises, and async functions

A future separates starting work from using its result. The caller gets the future at once, carries on, and later waits on it; the work runs meanwhile, on another thread or while the program waits on the network. Async functions (async and await in JavaScript and Python) are a way of writing code that works with futures as if it ran step by step: await pauses the function until the result is ready, and lets other work run in the meantime.

TripPlanner

Flight

Hotel

Car

start: future 1

start: future 2

start: future 3

100 ms: $95

200 ms: $420

300 ms: $310

All three run at once. The planner waits once, and the wait ends when the slowest answer arrives.

Language	Start a call	Wait for several	Runs on
Java	`CompletableFuture.supplyAsync()`	`join()` on each, or `allOf()`	A thread pool, the common pool by default
C++	`std::async(std::launch::async, ...)`	`get()` on each future	A thread per call
JavaScript	Call an async function: it returns a promise	`Promise.all()`, `Promise.allSettled()`	One thread; waiting happens in the event loop
Python	Call a coroutine; `asyncio.create_task()` to start it	`asyncio.gather()`, `asyncio.TaskGroup`	One thread; waiting happens in the event loop

Calling a coroutine or an async function in a sequence of awaits still runs the calls one after another, as the first version did. Concurrency comes from starting them all before waiting on any.

5 · Failures

When one of several fails

Waiting for several calls raises a question the sequential version never had: if one fails, what happens to the others? The common combinators answer differently:

Combinator	When one fails	The others
`Promise.all()`	Rejects at once with the first error	Keep running; nothing cancels them
`Promise.allSettled()`	Never rejects; reports each result or error	All finish
`asyncio.gather()`	Raises the first error, unless `return_exceptions=True` collects them	Keep running
`asyncio.TaskGroup`	Raises an `ExceptionGroup` once every task has ended; callers catch it with `except*`	Canceled as soon as one fails

Work that keeps running after nobody wants its result wastes resources, and an error in it can go unseen. Structured concurrency ties child tasks to the block that started them: the block doesn't end until every child has finished or been canceled, and a failure in one cancels its siblings. Python's TaskGroup works this way; in other languages, you cancel the remaining work yourself when the result is no longer needed.

6 · When breaking it is fine

Not everything should run at once

Concurrency has costs of its own:

Dependent calls must wait. If the hotel search needs the flight's arrival time, it can't start until the flight answers; forcing them together gives wrong results, not fast ones.
Load on the services. Starting a hundred calls at once can overwhelm a service that handled them one by one. Limit how many run together.
Harder failures. Sequential code fails at one clear point; concurrent code can have several failures at once, and they must all be reported or handled.

Keep calls sequential when each needs the previous result, when there are only one or two fast calls, or when the order of side effects matters, such as charging a card before booking the seat.

7 · Getting it right

Always wait, and always decide

Never drop a future. A future nobody waits on hides its errors; in C++, the future returned by std::async even blocks in its destructor, which can quietly make "concurrent" code sequential.
Decide each call's failure policy: required, so it fails the whole; optional, with a fallback; or retried.
Put timeouts on remote calls, so one stuck service can't hold the whole plan forever.
Don't block an event loop. In JavaScript and Python, a long computation inside an async function stops every other task; move it to a worker thread or process.

In Java, join() blocks the calling thread, which is fine on an ordinary thread. In a highly concurrent server, chain the next step with thenCombine() or thenApply() instead, or use virtual threads, where blocking is cheap.

8 · Look-alikes

Ways to wait for many things

Several tools wait for work that happens elsewhere.

Idea	What it gives you
Future or promise	One result that will arrive later
Callback	A function called with the result; the older style futures replaced
Thread pool	Where Java's and many other futures run their work
Producer-consumer	A stream of results through a queue, rather than one result per call
Reactive streams	Many results over time, with backpressure

9 · In practice

What the documentation shows

Python's asyncio documentation demonstrates this page's arithmetic: two coroutines awaited in turn take the sum of their waits, and started as tasks first they take the longer one. It also recommends TaskGroup over gather() because a failure cancels the remaining tasks. MDN notes that Promise.all() rejects with the first error without canceling the other operations.

Any page or API that gathers data from several services, such as a dashboard, a search across sources, or a checkout that prices shipping, tax and discounts, has this shape.

Check yourself

4 questions

1. Three independent calls take 200, 300 and 100 ms. Started together and awaited together, about how long does the planner wait?

2. const f = await flight.quote(); const h = await hotel.quote(); Do the calls run concurrently?

3. The flight call fails inside Promise.all(). What happens to the hotel and car calls?

4. The hotel search needs the flight's arrival time. Should the two calls start together?

Takeaways

Remember this

A future or promise is a handle to a result that arrives later; starting work and using its result become separate steps.
Start independent calls together and wait together: the total wait is the slowest call, not the sum.
Awaiting calls one after another is still sequential; concurrency comes from starting them before waiting.
Decide each call's failure policy; combinators differ on whether one failure stops or cancels the rest.
Never drop a future, put timeouts on remote calls, and don't block an event loop.