Low-level designDeep dive8 min

Object Pool

When objects are expensive to create and only a few are in use at once, keep a set of them ready, lend one out when needed and take it back afterwards. The catch: every object must be reset before the next borrower gets it.

Watch it evolve1 / 8
The problem

Drawing map tiles

TileRenderer draws each tile on a Canvas. A real canvas is several megabytes of pixel memory.

80
Start
Stage 1
Stage 2
Final

1 · The situation

A canvas for every map tile

A hiking app draws its map as square tiles. To draw a tile, TileRenderer needs a Canvas: in a real renderer that's a block of pixel memory several megabytes in size, slow to allocate and slow to free. The first version creates a fresh canvas for every tile and counts how many it made.

Here a canvas just records what was drawn on it, so the output can show it. Python writes the same method names; the other details match too.

The first versionJava
import java.util.ArrayList;
import java.util.List;

class Canvas {
    private final List<String> marks = new ArrayList<>();

    void draw(String mark) {
        marks.add(mark);
    }

    String contents() {
        return String.join(", ", marks);
    }
}

class TileRenderer {
    private int created;

    String render(String tile, List<String> features) {
        Canvas canvas = new Canvas();
        created++;
        for (String f : features) canvas.draw(f);
        return tile + ": " + canvas.contents();
    }

    int created() {
        return created;
    }
}

public class Main {
    public static void main(String[] args) {
        TileRenderer renderer = new TileRenderer();
        System.out.println(renderer.render("tile A", List.of("road", "river")));
        System.out.println(renderer.render("tile B", List.of("park")));
        System.out.println(renderer.render("tile C", List.of("road")));
        System.out.println("canvases created: " + renderer.created());
    }
}
«create»Canvas− marks: List<String>~ draw(String)~ contents(): StringTileRenderer− created: int~ render(String, List<String>): String~ created(): int

The class diagram of this version, drawn from its code: the renderer makes and discards a canvas per tile.

tile A: road, river
tile B: park
tile C: road
canvases created: 3

2 · The change

Thousands of tiles, one at a time

Panning across a map renders hundreds of tiles a second. Each one allocates a large canvas, draws on it for a moment, and throws it away, so the app spends much of its time allocating and collecting memory it immediately drops. Yet only one canvas is ever in use at a time.

A new canvas per tileJava
class TileRenderer {
    private int created;

    String render(String tile, List<String> features) {
        Canvas canvas = new Canvas();
        created++;
        for (String f : features) canvas.draw(f);
        return tile + ": " + canvas.contents();
    }

    int created() {
        return created;
    }
}
A canvas per tile
3
A pool
1

Canvases created to draw three tiles. With a pool, the count stays at the number in use at once, however many tiles are drawn.

The practice checks have nothing to say: the design is correct. The problem is cost, and it only shows when you measure.

3 · The fix

Lend canvases out, and take them back clean

Keep canvases that aren't in use, and hand one out when a tile needs it. The first attempt has a bug worth seeing:

A pool, first attemptJava
class CanvasPool {
    private final Deque<Canvas> free = new ArrayDeque<>();
    private int created;

    Canvas acquire() {
        if (!free.isEmpty()) return free.pop();
        created++;
        return new Canvas();
    }

    void release(Canvas canvas) {
        free.push(canvas);
    }

    int created() {
        return created;
    }
}
tile A: road, river
tile B: road, river, park
tile C: road, river, park, road
canvases created: 1

One canvas for all three tiles, and each tile shows everything drawn before it.

The pool works: one canvas is created and reused. But it comes back dirty. Tile B shows tile A's road and river, and tile C shows both. A pooled object carries its last borrower's state unless something resets it.

Canvas gains clear(), and CanvasPool.release() calls it before the canvas goes back on the free list. Every borrower gets a blank canvas.

Java
class CanvasPool {
    private final Deque<Canvas> free = new ArrayDeque<>();
    private int created;

    Canvas acquire() {
        if (!free.isEmpty()) return free.pop();
        created++;
        return new Canvas();
    }

    void release(Canvas canvas) {
        canvas.clear();
        free.push(canvas);
    }

    int created() {
        return created;
    }
}
*Canvas− marks: List<String>~ draw(String)~ contents(): String~ clear()CanvasPool− free: Deque<Canvas>− created: int~ acquire(): Canvas~ release(Canvas)~ created(): intTileRenderer− pool: CanvasPool~ render(String, List<String>): String
tile A: road, river
tile B: park
tile C: road
canvases created: 1

The tiles the first version drew, from one canvas.

4 · The pattern

Object Pool: borrow, use, return

Object Pool keeps a set of ready objects. A client acquires one, uses it, and releases it back. The pool creates new objects only when none are free, so the number created tracks how many are in use at once, not how many uses there are.

*Canvas− marks: List<String>~ draw(String)~ contents(): String~ clear()CanvasPool− free: Deque<Canvas>− created: int~ acquire(): Canvas~ release(Canvas)~ created(): intTileRenderer− pool: CanvasPool~ render(String, List<String>): String

The roles in the map app: CanvasPool lends canvases to TileRenderer and takes them back.

Role

In the map app

Its job

Pool

CanvasPool

Holds free objects, creates new ones when needed, resets returned ones

Reusable object

Canvas

Expensive to create; can be reset to a clean state

Client

TileRenderer

Acquires, uses and always releases

One tile's life: borrow a canvas, draw, give it back, and the pool resets it for the next tile.

5 · What it costs

Only worth it for objects that are truly expensive

  • Stale state. A pooled object that isn't reset leaks one borrower's data to the next, as the first attempt showed; with sensitive data, that's a security bug.
  • Leaks of a new kind. A borrower that never returns its object starves the pool.
  • Memory held forever. Pooled objects aren't freed while the pool lives.
  • Thread safety. With several threads, acquire and release need locking, or two threads may get the same object.

Pooling ordinary objects in a garbage-collected language usually makes things slower: modern allocators are fast, and long-lived pooled objects make collection more expensive. Pool objects that hold something costly beyond plain memory, and measure before and after.

Object

Pool it?

Why

A database or network connection

Yes

Opening one takes round trips, and servers limit how many can be open

A thread

Yes

Creating threads is slow; see Thread pools

A large buffer allocated many times a second

Often, after measuring

Allocation and collection costs add up

A small value object

No

Creating it is cheaper than managing a pool

6 · Getting it right

Reset on return, return in every case, cap the size

  • Reset in the pool, on release, so no borrower can forget.
  • Make returning automatic: a finally block, or a handle that returns the object when it goes out of scope.
  • Bound the pool. Decide what happens when every object is lent out: create more, wait, or fail.
  • Check before lending. A pooled connection may have died while it sat idle.
  • Never keep a reference after returning. The next borrower owns it now.

Language

Making the return automatic

Java

A try-finally, or a handle that implements AutoCloseable for try-with-resources

C++

A handle object whose destructor gives the object back (RAII)

JavaScript

A try-finally; resource management with using is newer and not yet everywhere

Python

A context manager, so with pool.borrow() as canvas: returns it on exit

7 · Look-alikes

Reuse in several forms

Several patterns reuse objects instead of creating new ones. What differs is whether the object is shared or lent, and whether it can change.

Pattern

Reused how

Can the object change?

Object Pool

Lent to one client at a time, then returned

Yes, and it's reset in between

Flyweight

Shared by many clients at once

No: it must be immutable

Singleton

One instance for everyone

Yes, which is part of its trouble

Cache

A computed result kept for later lookups

Usually not; it's replaced when stale

Pool and Flyweight are the pair to tell apart. A flyweight is shared by many users at the same time, so it must never change; a pooled object belongs to one user at a time, so it can change, as long as it's reset before the next.

8 · In practice

Pools you already rely on

Database drivers and web frameworks keep connection pools; thread pools run most server code; ASP.NET Core reuses string builders from a pool, and its pool resets each one on return. Each follows the same rules: borrow, use, return, reset.

The parking-lot practice problem is a pool in the physical world: a fixed set of spots, each lent to one vehicle and freed when it leaves. Design the parking lot and notice which object plays the pool, and what it means to return a spot clean.

Check yourself

4 questions

1. Why did tile B show tile A's road in the first pool?
2. A render throws halfway through, and the renderer never calls release(). What happens over time?
3. Which object is the best candidate for a pool?
4. How does a pooled object differ from a flyweight?

Takeaways

Remember this

  • Object Pool lends out reusable objects and takes them back, so creation tracks concurrent use, not total use.
  • Reset every object when it's returned, in the pool, so no borrower sees another's state.
  • Make returning automatic, and decide what happens when the pool is empty.
  • Pool objects that are costly beyond memory, such as connections and threads; measure before pooling anything else.
  • A pooled object is lent to one user at a time; a flyweight is shared by many and must not change.