Stateless services & sessions

A car-insurance site sells quotes through a five-step form: the car, the drivers, the address, the cover, then the price. It runs on one server, and that server keeps each visitor's progress in memory: who is logged in, and the answers so far. Leo has filled in steps 1 and 2.

Renewal season arrives, so the team adds a second server behind a load balancer, which takes turns sending requests to each. Leo submits step 3:

The vocabulary:

• State is anything a server remembers between one request and the next.
• A session is the state kept about one visitor, found again on each request through a session ID, usually stored in a cookie.
• A stateful server keeps state that only it holds. A stateless server keeps nothing between requests that another copy would need, so any copy can serve any request.

Stateless doesn't mean the system forgets. The state still exists; it just lives somewhere other than the server's memory.

The quickest fix is to make the load balancer send each visitor back to the same server every time. These are sticky sessions, also called session affinity. On the first request the balancer picks a server and sets a cookie naming it; later requests carrying the cookie go to that server. On Amazon's Application Load Balancer, for example, the cookie lasts anywhere from one second to seven days.

Leo's quote works again, as long as nothing changes. But servers change all the time:

The deploy row is the one teams hit first. A balancer can drain a server before removing it, letting in-flight requests finish (300 seconds by default on Amazon's balancer), but a quote form left open over lunch outlives any reasonable drain. Sticky sessions don't remove the state from the server; they only route around it while nothing changes (Load balancing).

The standard fix keeps the session ID in the cookie and moves the session itself to a shared session store, usually an in-memory key-value store such as Redis or Memcached. Every server reads it at the start of a request and writes it back at the end:

The session ID is now the key to someone's account, so it is built with care:

• Random and meaningless: at least 64 bits from a secure random generator, never a user ID or anything guessable.
• A protected cookie: marked Secure (HTTPS only), HttpOnly (page scripts can't read it) and SameSite (not sent on most requests started by other sites).
• A new ID after login, so an ID planted before login is useless afterward.
• Two expiries: an idle timeout, refreshed on every request (30 minutes here), and an absolute timeout after which the visitor logs in again whatever they are doing.

Sessions are small. 300,000 active sessions at 4 KB each is 300,000 × 4 KB = 1.2 GB: memory on one machine. The real cost is that the store is now on every request's path. If it goes down, every visitor is logged out at once, so it needs replicas and failover like any database (Database replication). And anything a visitor would be upset to lose, like a half-finished quote, is better saved as a draft in the database than kept only in a session that expires.

A second option keeps no session on the server at all. The server writes the session data itself into a signed token, a signed cookie or a JSON Web Token (JWT), and gives it to the browser. On each request, any server holding the signing key checks the signature and trusts the contents, with no lookup. Change one character and the signature no longer matches.

The revocation row decides most cases. If Leo's account is suspended at 10:00 and his token says it is valid until 10:15, every server will accept it until 10:15. The usual answer is short expiry with a way to get a fresh token, or a list of revoked tokens, and that list is shared state again. Keep token contents small and non-sensitive, and their lifetimes short. How login and tokens fit together is the subject of Authentication & authorization.

A server can be stateful without keeping a single session. The quote site has several other habits that break as soon as there is a second copy:

The rule of thumb: a server's memory and disk may hold anything that is safe to lose when the request ends, such as a temporary file or a cache. Anything that must outlive the request goes to a service built to keep it.

After the move, the quote site looks like this:

That buys almost everything else on the scaling path:

• Any balancing rule works, including least-busy rules that move a visitor from server to server (Load balancing).
• Autoscaling can add and remove servers without notice (Autoscaling).
• Deploys roll through the fleet one server at a time, and nobody loses a quote.
• A crash costs only the requests in flight, and those can be retried on another server.
• More servers mean more capacity, until the shared store becomes the bottleneck (Scalability).

That last point is the honest cost. Statelessness doesn't remove the hard part; it concentrates it. The web tier becomes easy to scale because the store and the database now carry all the state, and they are the parts that are hard to scale and must be replicated.

Where to keep what:

A checklist:

• Assume any request can reach any server, and test it: run two copies locally and alternate between them.
• Use sticky sessions only as a temporary crutch, and never as the only copy of anything that matters.
• Make session IDs random, cookies protected, and expiries real: idle and absolute.
• Replicate the session store, because its failure logs everyone out.
• Keep tokens small and short-lived, and decide up front how to revoke them.
• Hunt the hidden state: local files, scheduled jobs and in-memory counters.