Video streaming: transcoding & adaptive bitrate

An online fitness platform has 8,000 on-demand classes of about 45 minutes each, plus live classes that draw 20,000 viewers at once. The simplest design stores each class as one 1080p file at 6,000 kbit/s and lets the app download it: 6,000 kbit/s × 2,700 s ÷ 8 ≈ 2 GB per class.

That fails three kinds of viewers at once:

• On a train, with 2 Mbit/s, the video downloads three times slower than it plays and stalls every few seconds.
• On a phone screen, 1080p wastes data on pixels nobody sees.
• On fast Wi-Fi, the viewer gets one fixed quality even when a better one would fit.

And a viewer who jumps to minute 30 must wait for the download to get there.

Adaptive bitrate streaming solves all of it. Each class is encoded at several qualities (a ladder of renditions), each rendition is cut into segments a few seconds long, and a playlist lists them all. The player downloads one segment at a time and picks the rendition for the next segment from what it has just measured. Everything is plain HTTP files, so any CDN can serve them:

Compressed video stores most frames as differences from other frames. An I-frame (keyframe) can be decoded on its own; P- and B-frames describe only what changed. A group of pictures (GOP) starts at a keyframe, and playback can only start, or switch rendition, at one. HLS streams therefore use a keyframe every 2 seconds, segments that each start with one, and keyframes aligned across all renditions, so the player can switch at any segment boundary without a glitch.

The ladder itself trades bits for pixels. Apple's recommended H.264 ladder for 16:9 video:

Every rendition is stored, so the ladder multiplies storage: the nine rungs add up to 25,640 kbit/s, so one 45-minute class takes 25,640 × 2,700 ÷ 8 ≈ 8.7 GB, and 8,000 classes about 69 TB. Newer codecs reduce that: Apple's HEVC ladder reaches 1080p at 5,800 kbit/s instead of 7,800, about a quarter less, but older devices can't decode it, so services keep H.264 alongside newer codecs.

Transcoding is the expensive part, done once per upload. Because each segment begins with a keyframe and can be decoded on its own, a long video can be split into chunks that are encoded on many machines in parallel and joined afterward. Many services also tune the ladder per title instead of using one fixed ladder, because simple footage reaches the same quality with far fewer bits: a static instructor against a plain wall needs much less than fast cuts in a dance class.

The two dominant formats work the same way. Apple's HLS (HTTP Live Streaming) uses a multivariant playlist listing the renditions, and one media playlist per rendition listing its segments. MPEG-DASH, an international standard, describes the same structure in an XML manifest (an MPD). The multivariant playlist for one class looks like this:

Each media playlist then lists that rendition's segments, with a target duration that every segment must stay within. With Apple's recommended 6-second segments, a 45-minute class has 2,700 ÷ 6 = 450 segments per rendition, or 4,050 files across nine renditions, plus the playlists.

Historically HLS used one container format and DASH another, so services stored everything twice. CMAF (Common Media Application Format), standardized in 2018, defines fragmented MP4 segments that both an HLS playlist and a DASH manifest can point to, so one set of segments serves both kinds of player.

The server doesn't decide quality; the player does, one segment at a time. A viewer starts a class on a train:

Players combine two signals:

• Throughput-based rules estimate bandwidth from recent downloads and pick the highest rendition that fits. They react fast but are fooled by short bursts.
• Buffer-based rules, such as BOLA, pick quality from how many seconds are already buffered: a full buffer can afford a big segment, a nearly empty one can't. They are stable but slow to start.

The dash.js reference player switches between the two depending on the buffer level, and adds further rules, for example one that looks at the history of recent switches, because quality that flickers up and down is noticeable too.

Startup is its own decision. A high first rendition means a slow first frame; a low one means a blurry first few seconds. The usual first variant is 2,000 kbit/s on Wi-Fi and 730 kbit/s on cellular. Every one of these rules serves the goal adaptive streaming was designed for: the best quality that can still arrive in time, without stalling.

A popular live class with 20,000 viewers at an average of 3 Mbit/s needs 20,000 × 3 = 60 Gbit/s of delivery. No origin should see that. Segments are perfect cache content: immutable files with unique names, requested by thousands of viewers within seconds of each other.

Two CDN features matter most for live video. Request collapsing consolidates simultaneous requests for the same brand-new segment into as few as one request to the origin. An origin shield, an extra cache layer in front of the origin, means every edge and regional cache fetches from it instead of from the origin, so each segment leaves the origin about once. Storage and lifecycle rules for the files themselves are in Object & blob storage.

A live class adds a clock. The camera feed goes to a live encoder that produces the ladder in real time, the packager publishes each segment as soon as it is complete, and players follow the growing playlist. HLS players don't start closer than three target durations to the end of a live playlist, so the delay behind real time is at least three segments plus encoding and delivery:

For a fitness class, an 18-second delay is awkward: the instructor answers a chat question long after it was asked. Low-Latency HLS keeps 6-second segments for normal playback but also publishes partial segments, for example 200 ms each, as soon as they are encoded. Players hold an open request for the next playlist update instead of polling (blocking playlist reload), and fetch hinted parts before they finish. The resulting latency is in the range of standard television broadcasts.

The cost is load: a player that fetched one segment every 6 seconds now fetches a part every 200 ms, and every edge server must handle requests that wait for media that doesn't exist yet. For a broadcast to thousands, LL-HLS is a good trade. For a truly interactive session, such as a personal trainer correcting your form, segment-based streaming is the wrong tool, and real-time protocols built for calls are used instead.