Reverse proxies & API gateways

A fitness app has a website and a phone app. Behind them run a handful of servers, and nobody on the internet should talk to those servers directly. Instead every request goes to one machine at the edge, which passes it on and passes the answer back. That machine is a reverse proxy: it answers for the servers, and clients never learn they exist.

Because every request passes through it, the reverse proxy is the natural place for work that every server would otherwise repeat:

• Terminate TLS: decrypt HTTPS once at the edge, so servers don't each manage certificates.
• Compress responses and cache files that don't change, like images and scripts.
• Hide the servers: their addresses and software never face the internet, and obviously bad requests are dropped before reaching them.
• Spread the load across them, which makes it a load balancer too (see Load balancing).

The fitness app grows. Profiles, workouts and the friends' leaderboard each become their own service, with its own servers and its own team. Now the phone would need three addresses, and every time a service is split or renamed, every app would have to change.

An API gateway fixes that by being the single entry point for all of them. It reads each request, and its path decides which service gets it. Clients keep one address; behind it, services can be added, split or moved, and only the gateway's routing table changes.

A gateway still runs as several copies for capacity and safety, so in practice a load balancer sits in front of the gateway copies, and the gateway in turn spreads requests across each service's servers. A system with one or two services rarely needs a gateway at all: a load balancer is enough.

Every service needs the same chores done: check who is calling, stop callers who send too much, record what happened. Done in each service, they are written several times, in several languages, slightly differently. The gateway can do them once for everyone.

Two rules keep this safe:

• Shared jobs only, never business logic. "Is this token valid?" belongs at the door. "Can this user see that workout?" belongs in the workout service, which knows the rules.
• No way around the door. Services must accept traffic only from the gateway, and the gateway must throw away any identity headers a client sent itself before adding its own. Otherwise anyone can skip the checks, or claim to be someone else.

The app's home screen shows your profile, today's workout and where you stand among friends: data from three services. Say each round trip from a phone takes 100 ms. Calling the services one after another costs 300 ms of waiting before the screen can draw, and every call is another chance for a patchy mobile connection to fail.

The gateway can take one request and make the three calls itself, all at once and close to the services, where a round trip is a fraction of a millisecond. This is aggregation.

The combined answer waits for the slowest of the three. So give each inside call a timeout, and decide in advance what happens when one misses it: the home screen can show profile and workout without the leaderboard, but a checkout cannot skip the payment. Keep the gateway's part small; heavier combining belongs in a service of its own behind it.

The phone wants small answers and few calls; the website shows far more on one page. A single gateway that serves both collects every client's special cases, and every team must change it and wait for it. It slowly becomes the monolith the services were split up to avoid.

A common answer is a backend for frontend: one gateway per kind of client, owned by the team that builds that client. The phone team shapes the phone's answers without waiting for anyone. The cost is some duplicated combining code across the gateways, which is cheaper than a shared one that nobody can change quickly.

Everything that makes a gateway useful comes from every request passing through it. That is also where its costs come from.

So the decision is simple to state. One or two services and one kind of client: a load balancer, or a plain reverse proxy, is enough. Several services, several kinds of client, API keys to check or abuse to stop: put a gateway at the door, and plan from day one for it to be copied, sized for peaks, and kept thin.