WebRTC & peer-to-peer

An online tutoring app runs 10,000 one-to-one video lessons at the evening peak, plus group study rooms of up to 12 students. The first design sends every stream through the app's servers: the student's camera goes up to the data center and back down to the tutor.

That costs twice over:

• Distance. A student and tutor in the same city still send every frame to a distant data center and back, adding round trips to a conversation where every 100 ms is felt.
• Bandwidth. At about 1.5 Mbit/s per video stream, each lesson pushes two streams through the server: 10,000 × 2 × 1.5 Mbit/s = 30 Gbit/s at peak, in and out, paid for by the app.

WebRTC (Web Real-Time Communication), built into every major browser and available to native apps, lets the two devices send media straight to each other. The app's servers only introduce them, and step back in only when a direct path is impossible or a call has too many people. Most of this page is about the two obstacles: devices that can't be reached directly, and groups.

The student's laptop has an address like 192.168.1.23, valid only inside the home network. The home router shares one public address among every device in the house using NAT (network address translation, covered in Private networks: NAT, VPCs & firewalls): outgoing traffic is rewritten to the router's public address and a port, and replies to that traffic are let back in. Traffic that nobody inside asked for is dropped:

So a peer-to-peer call needs three things before any video flows: a way for the two devices to exchange messages before they are connected, a way for each to learn the public address its router uses, and a fallback relay for the routers and firewalls that won't cooperate. WebRTC builds these from standard pieces.

Before two devices can talk directly, they must agree on what to send (codecs, resolutions, encryption keys' fingerprints) and how to reach each other. They exchange this through a signaling server, which WebRTC deliberately leaves unspecified: a WebSocket server is typical (WebSockets & long polling). The descriptions are written in SDP (Session Description Protocol), a data format, not a transport:

The signaling server still matters for security. It sees and forwards the fingerprints that the peers use to authenticate each other's encryption keys, so a malicious signaling server could insert itself into the call. WebRTC has identity mechanisms for applications that need protection even from their own signaling server.

Each device gathers candidates, possible addresses it could be reached at, and sends them to the other through signaling. ICE (Interactive Connectivity Establishment) then tests pairs of candidates and picks the best one that works:

STUN is cheap: a STUN server answers one small question per device and never touches the media. TURN is expensive: it is the relay design from the first section, used only for the calls that need it, such as routers that pick a different public port for every destination (symmetric NAT) and corporate firewalls that block UDP. For the last case, WebRTC implementations must support TURN over TCP and over TLS, which can get through networks that allow little more than web traffic.

Plan TURN capacity from your own measurements rather than a rule of thumb: the share of calls that need a relay depends heavily on whether users are at home, on mobile networks or behind corporate firewalls. Every relayed lesson costs the same 3 Mbit/s through your servers as the original design, and TURN servers should be close to users, because a relay on another continent adds its round trip to every packet.

A video frame that arrives 2 seconds late is worse than one that never arrives: the conversation has moved on. So WebRTC media normally runs over UDP, which delivers now or never, instead of TCP, which holds back everything behind a lost packet until it is resent (How data moves across a network).

Packets still arrive with uneven gaps. The receiver holds them briefly in a jitter buffer to play them out smoothly. The buffer can only absorb as much variation as the delay it adds, so it is a direct trade: a bigger buffer means fewer glitches and a longer delay before the student hears the tutor.

Three more properties come built in:

• Encryption is mandatory. All media must use SRTP with keys set up by DTLS; WebRTC implementations may not negotiate unencrypted media.
• Bitrate adapts. Endpoints run congestion control and set each codec's send rate to its share of the bandwidth currently available, so quality drops instead of the call freezing, much as adaptive streaming does for recorded video (Video streaming: transcoding & adaptive bitrate).
• Data channels carry other things, such as a shared whiteboard's strokes, over the same connection, in reliable or partially reliable modes.

In a pure peer-to-peer mesh, everyone sends their video to everyone else. With N people, each device uploads N − 1 copies of its stream, and home upload links are usually far smaller than download links:

A selective forwarding unit (SFU) receives one stream from each participant and forwards it to everyone else, without decoding or mixing. Senders can upload a few qualities at once (simulcast), and the SFU forwards each receiver the quality it can handle: a student on a weak connection gets the small version, without making everyone else's worse. The older alternative, an MCU, decodes and mixes all streams into one, which saves the receivers bandwidth but costs the server heavily in CPU and adds delay.

The tutoring app ends up with both: one-to-one lessons go peer to peer, falling back to TURN when they must, and group study rooms go through an SFU from the start. Download still grows with the room, since each person receives N − 1 streams, so large rooms show only the active speaker and a few thumbnails at full quality.

Peer-to-peer removes the media from your servers, not the servers from your system. The tutoring app still runs:

Anything that must outlive the call belongs on a server: who attended and for how long, chat messages, shared files and recordings, which means some media goes through a server after all when a lesson is recorded. Monitor calls from the client side too: connection setup time, the share of calls that fall back to TURN, packet loss and freezes are invisible from the server when media never touches it.