Private networks: NAT, VPCs & firewalls

A bookkeeping service for small businesses runs a load balancer, a dozen app servers and a PostgreSQL database holding every customer's accounts. Only one of those needs to be reachable from the internet: the load balancer. The database, especially, should be impossible to even try to connect to from outside.

The tool is private addresses: ranges reserved for internal networks, which internet routers are expected to filter out. Traffic addressed to them never crosses the public internet. Three ranges were reserved in 1996 and are still used everywhere, from home Wi-Fi to cloud networks:

Private addresses were also an answer to scarcity: IPv4 has only about 4.3 billion addresses in total, far fewer than the devices that want one. Any number of private networks can reuse the same ranges, as long as something translates at the edge.

When an app server at 10.0.2.15 calls the bank's API to import a customer's transactions, the bank must be able to reply. NAT (network address translation) rewrites the outgoing packet's source to a public address and a port it picks, remembers the mapping, and translates the reply back:

Three consequences matter in design:

NAT is also why peer-to-peer calls are hard (WebRTC & peer-to-peer) and why servers that must accept connections need a public address or a load balancer in front. With IPv6 there are enough addresses for every device, and clouds offer an egress-only gateway that allows outbound connections without NAT while still refusing unsolicited inbound ones.

In the cloud, the bookkeeping service gets a virtual private cloud (VPC): a network logically isolated from every other customer's, with an address range it chooses, such as 10.0.0.0/16 (65,536 addresses). Inside it:

• Subnets are smaller ranges, such as 10.0.1.0/24 (256 addresses). In AWS each subnet lives in one availability zone; in Google Cloud the VPC is global and subnets are regional.
• Route tables decide where traffic for each destination range goes: stay inside the VPC, go to the internet gateway, go to a NAT gateway, or go to a connected network.
• An internet gateway connects the VPC to the internet for the subnets whose routes point to it.

Whether a subnet is public or private is decided by one route. A public subnet has a route for all internet traffic (0.0.0.0/0) to the internet gateway, and its machines can have public addresses. A private subnet has no such route; its machines can reach the internet only through a NAT gateway sitting in a public subnet, and nothing on the internet can start a connection to them.

The bookkeeping service splits its tiers by subnet, repeated in each availability zone for redundancy (Redundancy & failure domains):

Each tier can only be reached from the one in front of it. An attacker who finds a bug in the load balancer's configuration still can't connect to the database directly, because there is no route from the internet to its subnet and no rule that would admit them. That layering is defense in depth: several independent barriers, any one of which might fail on its own.

Routes decide where traffic can go; firewall rules decide whether it is allowed. Clouds provide two kinds, and AWS's are typical:

Security groups can name other security groups instead of addresses, which matters when addresses change with every deploy (Service discovery & registration). The bookkeeping service's rules read like the architecture itself:

Keep rules as narrow as the architecture allows: specific ports, specific source groups, no "allow all from 0.0.0.0/0" on anything but the front door. Google Cloud's firewall works the same way: stateful, with ingress denied unless a rule allows it and egress allowed by default.

Two kinds of middleman sit at the edges of a private network, facing opposite directions:

Controlling egress is easy to forget and valuable: it limits where stolen data can be sent and stops a server from being tricked into calling internal addresses (Webhooks covers this for user-supplied URLs). Cloud services can be reached without the internet at all: private endpoints, such as AWS PrivateLink, make a cloud service appear at a private address inside the VPC, so traffic to object storage or a queue never needs the NAT gateway.

Private networks also connect to each other. VPC peering joins two VPCs; a transit gateway joins many; a site-to-site VPN or a dedicated line connects a VPC to an office or data center. Each connection is another path an attacker could use, so give it the same narrow rules as anything else.

Administrators still need to reach private servers. The old answer is a bastion host, one hardened machine in a public subnet that admins log in to first; managed tools such as AWS Systems Manager Session Manager open sessions through the provider's own service, with no inbound port, bastion or SSH key to manage, and log every session. Either way, admin access should be logged, short-lived and tied to a person (Network segmentation & zero trust goes further).