Edge computing

A fashion store runs its application and database in Frankfurt and sells in 40 countries. Its CDN already serves images and scripts from servers near each shopper (Content delivery networks (CDNs)). But every request that needs a decision, such as "send this Australian shopper to the Australian store", still goes to Frankfurt.

From Sydney that is far. The great-circle distance to Frankfurt is about 16,500 km, and light in fiber costs about 1 ms of round trip per 100 km (Latency), so each round trip takes at least 165 ms, before any handshakes or the longer real cable routes. A redirect that the origin decides in a microsecond still costs the shopper that trip.

Edge computing runs code on the CDN's own servers, in the location that received the request. Cloudflare, for example, lists data centers in 348 cities, and runs its Workers code in the one closest to where a request arrived by default. The redirect is decided in Sydney; only requests that really need Frankfurt go there.

An edge location runs code for many customers on few machines, and may see a given customer's code only occasionally, so starting fast and staying small matter more than anything else. Cloudflare Workers runs code in V8 isolates, the same sandboxes a browser uses to keep web pages apart: many isolates share one runtime process, each with its own memory. Cloudflare's isolates start in about 5 ms, around a hundred times faster than a Node process, using an order of magnitude less memory.

The price is a narrower environment. AWS offers two edge options on its CDN, and their documented limits show the trade clearly:

Cloudflare's Workers sit between the two: 128 MB of memory, and CPU time per request limited to 10 ms on the free plan and up to 5 minutes on paid plans, with network calls allowed. Across all of them, the rule is the same: edge code should do little work per request, quickly, and keep no state of its own between requests, since an isolate can be evicted at any time.

Work that can be decided from the request alone is ideal. AWS lists exactly these uses for its lightweight edge functions, and the fashion store has one of each:

• Redirects and rewrites: send / from an Australian IP to /au/.
• Request authorization: verify the signature on a JSON Web Token, rejecting bad requests before they cross the ocean (Authentication & authorization).
• Header manipulation: add security headers to every response.
• Cache key normalization: strip tracking parameters so equal pages share one cache entry, raising the hit rate.

A step further, the edge can assemble pages: fetch a cached product page and insert the shopper's country, currency or experiment group, so a mostly static page needn't come from the origin just because one part varies. Assigning experiment groups by hashing a user ID works well here, since it needs no data at all (Feature flags & configuration).

The store's developers move the cart page to the edge. The page needs three queries in sequence: the cart, then the prices for its items, then stock for those items. Each query from Sydney to the Frankfurt database is a full round trip:

Moving computation toward the user only helps if the data it needs is there too. Otherwise the code belongs next to the data. Cloudflare's Smart Placement solves exactly this case: it watches a Worker's traffic and runs it near the back end it calls most, instead of near the user.

So split the work. The edge handles what needs only the request, and passes everything else to code running beside the database, ideally in one call that does all three queries there. Or bring some data to the edge, which the next section is about.

Edge platforms offer data stores, and they make the trade-offs of distributed data very visible (Consistency models):

Workers KV is fast to read because it is cached, which also makes it eventually consistent. A change is usually visible at once in the location where it was made, but other locations may keep serving their cached copy for up to 60 seconds or more. Even a lookup that found nothing is cached, so a newly created key can stay "missing" elsewhere for the same time:

Durable Objects take the opposite trade. Each object has a globally unique name and lives in one place, created close to where it was first requested, with storage of up to 10 GB that is strongly consistent and transactional. Requests from anywhere reach that one instance, which handles them one at a time. A shopper's cart as one object is consistent and fast for that shopper; a single global stock counter as one object would make every shopper in the world wait in one line.

Edge platforms push new code to every location within seconds, which is part of their appeal. It is also the risk. On July 2, 2019, Cloudflare deployed a new web application firewall rule containing a regular expression that backtracked enormously. It exhausted the CPU that serves HTTP traffic on every core handling it, across its network worldwide, and took down its proxying, CDN and firewall for 27 minutes.

Cloudflare's own list of causes included that a non-emergency rule change could go globally into production without a staged rollout, and that rolling back took too long. The lessons carry over to any edge code:

• Stage edge deploys like any other release: a few locations, then more (Deployment strategies).
• Keep CPU limits per request, so one bad change can't take a whole machine.
• Keep a fast way back, tested, and a way to reach your own systems when the edge is down.
• Watch per-location metrics, since a problem may start in one place.

For the store, the checklist:

• Count round trips to data before moving code to the edge; one distant query per request is the most an edge function should make.
• Keep edge code small and stateless, within CPU and memory limits.
• Use eventually consistent edge stores for what can be a minute stale, and a single-owner or origin store for what can't.
• Place code near its data when it must make several calls.
• Stage edge deploys and keep a tested rollback.