Scalability

A food-ordering app launches in one city. A thousand orders on a Friday night, one server, no trouble. Then it expands to the whole country, and Friday night brings a hundred times the orders. Does it cope?

Scalability is a system's ability to handle more load by adding resources. The ideal: twice the machines, twice the capacity. It is a different question from speed:

• A performance problem: the app is slow even for one user.
• A scalability problem: the app is fast for one user, and slow when ten thousand arrive.

"Scalable" also needs an object. Scalable to what? Orders per second, people browsing at once, restaurants listed, photos stored: each is a different load, and a design can handle one well and another badly. Before you say a design scales, say which number it scales with.

There are two ways to add capacity. Vertical scaling, or scaling up, means a bigger machine: more processors, more memory. Horizontal scaling, or scaling out, means more machines sharing the work.

Do not underestimate one machine. In 2026 the largest cloud servers offer up to 1,920 virtual processors and 32 TB of memory. Plenty of successful products never outgrow a few well-sized machines, and one machine never has to coordinate with anyone. So a common path is: scale up first, because it is simple, and scale out when a number demands it, or when one machine's failure would be too costly.

Scaling out has a trap. Suppose each server keeps your cart in its own memory. The load balancer does not know that:

Two ways out:

• Sticky sessions. The load balancer always sends you to server A. It works until A dies (your cart dies with it) or until A gets more than its share of busy customers.
• Stateless servers. No server keeps anything between requests. The cart lives in a shared store every server can reach, so any server can answer any request.

Notice what happened to the state: it did not disappear, it moved into the shared store. That is the pattern of almost every scalable system. The servers become easy to copy, and all the difficulty collects in the data layer, which later pages tackle with copies and splitting.

If servers are stateless, can you add them forever? No. Somewhere, every request touches something shared: one database table, one lock, one counter that hands out order numbers. That part can only do one thing at a time, and it caps the whole system.

Amdahl's law puts a number on the cap. If a share s of each request's work must happen one at a time in a shared place, then no number of servers can give you more than 1 ÷ s times the capacity of one. With s = 5%, that is 1 ÷ 0.05 = 20 times, whether you run 30 servers or 3,000.

Real systems are worse, because servers also have to keep each other up to date: clearing stale cache entries, agreeing on who is alive, passing changes along. Every server may need to hear from every other, and the number of pairs grows much faster than the number of servers. The universal scalability law adds that cost:

capacity with N servers = N ÷ (1 + a(N − 1) + bN(N − 1))

• a is contention: the share of work that waits for something shared (Amdahl's cap).
• b is coherency: the cost of keeping the servers consistent with each other.

That falling line is the surprise: past a point, adding servers makes the system slower. It is not a theory curiosity; it is what happens when a design has every node talk to every other.

So scaling work is mostly hunting for shared things and removing them. In the food app they look like this:

• One database for every write. Every order in the country lands in the same place.
• One counter. Order numbers handed out one at a time, by one machine.
• One hot item. The most popular restaurant at lunch: thousands of orders hitting the same records.
• Chatty servers. Every server telling every other when a menu changes.

Growing from one city to a country is a sequence of these fixes, each triggered by a number:

Each step removes one shared thing, and the next bottleneck appears somewhere else. The tools named here each get their own page later: load balancing, caching, replication and sharding.