Privacy, residency & retention

A recruiting platform connects job seekers in Europe and North America with employers. Lena, an applicant in Berlin, finds a job and writes: please delete everything you hold about me. The account row takes a second to remove. Then the team starts counting where else Lena's data lives:

Each copy was made for a good reason, and none of them was designed with deletion in mind. That's the pattern this page is about. Personal data is any information relating to an identified or identifiable person; the EU's General Data Protection Regulation (GDPR) uses that definition, and it covers ids and online identifiers as well as names. Laws like it turn a handful of principles into engineering requirements. Other jurisdictions have similar rules: California's, for example, gives consumers rights to know and to delete.

The GDPR's principles and rights translate fairly directly into things a system must be able to do:

The deadlines are short. A request like Lena's must be answered without undue delay and within one month, extendable by two further months for complex or numerous requests. A personal data breach must be reported to the regulator without undue delay and, where feasible, within 72 hours of becoming aware of it, unless it is unlikely to put people at risk. The largest fines reach €20 million or 4% of worldwide annual turnover, whichever is higher.

Erasure isn't absolute. The right gives way where keeping the data is necessary to comply with a legal obligation, or to establish, exercise or defend legal claims, among other exceptions. So a deletion system needs exceptions as data, not as ad hoc judgment: Lena's invoices for a paid premium plan may have to stay, while everything else goes.

The GDPR requires a record of processing activities: for each purpose, the categories of people and data, who receives it, any transfers abroad and, where possible, the time limits for erasure. Kept as a document, it goes stale within months. Kept as part of the system, it becomes the map that deletion, export and retention run on:

The practices that make the map real:

• Tag personal fields in the schema, so tooling can find them, mask them in logs and include them in exports.
• Carry one subject id everywhere, so a single key finds a person in every store, instead of matching names and emails.
• Derive copies through known paths, such as a change stream (Change data capture), rather than ad hoc exports nobody tracks.
• List outside recipients, such as the email provider: when data has been disclosed to others, each must be told about the erasure, unless that proves impossible or takes disproportionate effort.

Pseudonymization helps without changing the legal status. Replacing Lena's email with a random id in the warehouse means analysts never see who Lena is. But pseudonymous data that can be linked back with additional information is still personal data; only data that is truly anonymous, where nobody can be identified by any means reasonably likely to be used, falls outside the rules. A warehouse keyed by a reversible id still needs deletion.

A request to delete subject 7731 becomes a job with a checklist, not a single SQL statement. A privacy service records the request, works out which exceptions apply, fans the deletion out to every system in the map, and only reports success when each one has confirmed:

Backups are the hard case, because rewriting old snapshots is slow and risky. Erasure must reach backups too, but regulators accept that data may stay in a backup until it is overwritten on its normal schedule, provided it is put beyond use: not used for any other purpose, and the person is told clearly what will happen. One consequence is easy to miss: restoring a backup brings Lena back, so every restore has to replay the deletions recorded since that backup was taken.

Crypto-shredding is the other way through immutable copies: encrypt each person's data with their own key and delete the key, which makes every copy unreadable at once, backups included (Encryption in transit & at rest). Event logs that must never be edited use the same trick (Event sourcing & CQRS). It works only if the key itself is never copied into the backups it is meant to cut off.

Whether destroying a key legally counts as erasure is still debated: regulators accept it in some cases as putting data beyond use, so check with whoever answers for compliance before relying on it alone.

Most personal data isn't deleted because someone asked; it's deleted because its purpose ended. Without a rule, it never is. Suppose the platform gains 100,000 applicants a month and keeps everything forever: after three years it holds 3.6 million profiles, most of people who stopped looking long ago. A rule that deletes a profile 12 months after its last activity caps the total:

A retention schedule gives each class of data a period and a reason, set by the legal team and enforced by the system. For the platform, the schedule might read:

The periods above are the example's policy, not legal advice: the right period depends on the purpose and the jurisdiction. The mechanics are covered in Data lifecycle: tiering, TTL & compression: a TTL on rows and objects, lifecycle rules on storage, and partitions dropped whole. Two rules make them trustworthy: the deletion job is monitored like any other job, and a legal hold is a flag the job checks, not a manual pause.

A large German employer asks: is our applicants' data kept in the EU? Under the GDPR, personal data may be transferred outside the European Economic Area only under the regulation's transfer rules: to a country the European Commission has found adequate, or with appropriate safeguards such as the Commission's standard contractual clauses. Remote access from a third country, for example during support, also counts as a transfer.

The rules can shift under a running system. In July 2020 the EU's Court of Justice invalidated the adequacy decision for the EU-US Privacy Shield. In May 2023 Ireland's regulator fined Meta €1.2 billion for continuing to transfer Facebook users' data to the US on standard contractual clauses after that ruling, and ordered it to suspend the transfers. In July 2023 the Commission adopted a new adequacy decision, the EU-US Data Privacy Framework, for US companies that participate in it.

For a system, residency means a regional deployment per jurisdiction, with each person's data placed in one region and a thin global layer that holds as little as possible:

The database is rarely what breaks residency. The usual leaks are the copies and accesses around it:

Cloud providers let you choose the regions where data is stored, and AWS, for example, commits not to move content out of the chosen regions without agreement, except as needed to provide the service or comply with the law. Everything you build on top is your own responsibility. In a multi-tenant product, a region is usually a separate deployment stamp (Multi-tenancy).

The GDPR calls this data protection by design and by default: measures built in when the system is designed, and defaults that collect, keep and expose only what each purpose needs. For the recruiting platform: