System designCore8 min
How data moves across a network
A photo leaves your phone as two thousand separate pieces and arrives whole on the other side of the world. Here is what addresses it, carries it, and puts it back together, and when you would rather it didn't bother.
1 · The problem
A photo travels as two thousand separate pieces
You send a 3 MB photo to a friend. The network does not carry "a photo". It carries packets: small chunks of at most about 1,500 bytes on most links, each with the destination written on it. After the labels, each carries a little under 1,500 bytes of your photo, so the photo leaves as about 2,000 packets.
Each router looks at a packet's destination and passes it one step closer. No router knows the whole route, and packets from the same photo may take different paths.
Along the way, anything can happen to a packet:
- It can be lost, when a busy router has no room for it.
- It can arrive out of order, behind a later packet that took a faster path.
- It can arrive late, after sitting in a queue.
The network only promises to try. Everything else, from addressing to putting the photo back together, is built in layers on top of that one promise.
2 · Addresses
An address finds the machine; a port finds the program
Every device on the internet has an IP address. The older form, IPv4, is a 32-bit number written like 203.0.113.7, which allows about 4.3 billion addresses: fewer than the devices that want one. The newer IPv6 uses 128 bits, written like 2001:db8::7, which is more than enough for everything. Your home router usually shares one public address among all your devices.
An address gets a packet to the right machine. But that machine runs many programs at once. A port, a number from 0 to 65535, says which program the packet is for. Some are well known: 443 for HTTPS, 80 for plain HTTP, 22 for remote login. Think of the address as the building and the port as the apartment number.
Who | Address | Port |
|---|---|---|
Your phone | 192.0.2.10 | 51544, picked at random for this conversation |
The server | 203.0.113.7 | 443, because it speaks HTTPS |
Those four numbers, two addresses and two ports, together with the protocol, identify one conversation. That is how a server tells apart thousands of people talking to port 443 at the same time.
3 · Layers
Each layer solves one problem and trusts the one below
Networking is split into layers. Each one handles a single job and wraps the data from the layer above in its own envelope, with its own label on the outside:
Each router unwraps only as far as the IP label, decides the next hop, and wraps the packet in a new frame for the next link. Only the two ends ever open the inner envelopes.
Layer | Its one job | Examples | OSI number |
|---|---|---|---|
Application | What the programs say to each other | HTTP, DNS, email | 7 (with 5 and 6) |
Transport | Between two programs: ports, and reliability if wanted | TCP, UDP | 4 |
Internet | Across networks, hop by hop, best effort | IP | 3 |
Link | To the next device on the same wire or radio | Ethernet, Wi-Fi | 2 (with 1) |
The internet actually runs on these four layers. A second, older model, the seven-layer OSI model, split things more finely, and its numbers stuck. That is why engineers say "a layer 4 load balancer" (it sees only addresses and ports) or "a layer 7 load balancer" (it reads the HTTP request, so it can route by URL). Knowing the numbers is mostly knowing what each device can see.
4 · Transport
TCP delivers everything in order; UDP delivers now or never
The transport layer offers two very different deals. TCP turns lost, jumbled packets into a clean stream: it numbers every byte, the receiver confirms what it got, and anything missing is sent again. It also slows down when the network looks congested, so it does not make things worse.
Nothing is lost and nothing is out of order, but packet 3 had to wait for packet 2's second trip. One lost packet holds up everything behind it.
UDP makes no promises at all: each packet goes out on its own, with no connection, no order and no resending. Why would anyone want that? Because sometimes late data is worthless. In a video call, the sound of a word from half a second ago is useless; better to skip it and play what is arriving now than to freeze while it is resent.
What | TCP | UDP |
|---|---|---|
Before sending | A handshake: one round trip | Nothing: just send |
Lost packets | Sent again, automatically | Gone, unless the program copes |
Order | Always the order sent | Whatever order they arrive |
Good for | Web pages, files, email, databases | Video and voice calls, live streams, games, quick lookups such as DNS |
The line is not rigid. HTTP/3 runs over UDP and builds its own reliability on top, so it can avoid one lost packet holding up every other stream on the connection.
5 · In design
Where this shows up when you design a system
Three habits come straight out of this page:
- Connections are not free. Every new TCP connection costs a round trip before any data moves, and a TLS round trip on top. Reuse them (see What happens to one request).
- Choose the transport by what late data is worth. If a stale packet is useless, as in live audio, video or game positions, UDP lets you drop it. If every byte matters, TCP.
- Know what each box can see. A device working at layer 4 sees addresses and ports; one at layer 7 reads the request itself. That decides what it can route, cache or block.
A box working at | Sees | So it can |
|---|---|---|
Layer 4 | Addresses and ports | Spread connections over servers, and allow or block them by address and port |
Layer 7 | The request itself: path, headers, cookies, once TLS is decrypted | Route by path, cache responses, and block requests by their content |
Check yourself
5 questions
Takeaways
Remember this
- Data crosses a network as small packets, each routed on its own, and any of them may be lost, reordered or delayed.
- The IP address finds the machine; the port finds the program on it.
- Layers each do one job: link to the next hop, IP across networks, TCP or UDP between programs, HTTP and others on top.
- TCP gives a complete, ordered stream at the cost of a handshake and waiting for lost packets; UDP sends now and never waits.
- "Layer 4" devices see addresses and ports, "layer 7" devices read the request: that decides what they can do.