System designCore9 min
Long-running & bulk APIs
Some requests take minutes: importing a big file, exporting an account, processing thousands of items at once. Holding a connection open that long fails in ways nobody controls. The fix is to accept the work at once, hand back a job the client can check on, and deliver large results as a file.
1 · The problem
Some work takes longer than any connection should stay open
A newsletter platform lets customers upload subscriber lists. A typical upload is a CSV of 200,000 rows, and each row must be checked, deduplicated and saved. At about 1,000 rows a second, that's 200 seconds of work. If the upload request waits for all of it, the request has to survive every timeout between the customer's browser and the database:
A rule of thumb: use a long-running operation for anything that may take more than about 10 seconds. A proxy in front of the servers gives up on a silent request after 125 seconds by default, long before the import is done.
Even when nothing times out, a long request goes wrong in other ways:
- The connection drops. A laptop lid closes or a phone changes networks, and the client can't tell whether the import happened.
- Retries duplicate work. A client that times out and tries again starts a second import of the same file.
- Servers are held hostage. A server busy with one 200-second request can't be restarted or scaled down without breaking it.
2 · Accept, then finish
Accept the work now, and give the client a job to check on
HTTP has a status code for exactly this. 202 Accepted means the request was accepted for processing but isn't finished. The response should describe the current status and point to a status monitor: a separate URL the client can check until the work is done.
The upload request ends in under a second. The 200 seconds of work happen in a worker (Message queues & pub/sub), and the client checks in on its own schedule, from any connection.
The job, not the connection, now carries the state. The browser can close, the API server that took the upload can be replaced, and the customer can come back tomorrow: the status monitor still answers. Microsoft's APIs return its address in an Operation-Location header; Google's return an Operation object whose name is used to look it up.
3 · The job resource
A job is a resource with a status, progress and a result
The status monitor is a small resource of its own. The newsletter platform's looks like this while it runs:
GET /imports/imp_8f3k
{
"id": "imp_8f3k",
"status": "Running",
"createdAt": "2026-10-02T09:14:03Z",
"progress": { "rowsDone": 90000, "rowsTotal": 200000 },
"result": null,
"error": null
}Large API platforms settle on the same parts:
Part | What it holds |
|---|---|
Status | A fixed set of values: not started, running, succeeded, failed, canceled; or a single "done" flag |
Metadata | Progress and details while running, such as rows done out of rows total |
Result | On success, the outcome or a link to it, such as the new list's ID |
Error | On failure, what went wrong, in the same format as any other API error |
Two more decisions belong to the job's lifecycle:
- How long it lives. Clients may check in late, so keep finished jobs a while: at least 24 hours, and 30 days is common.
- Whether it can be stopped. A cancel action (
POST /imports/imp_8f3k:cancel) lets a customer stop an import of the wrong file. The worker checks for cancellation between batches, and the job ends as canceled, with what was already saved reported.
4 · Finding out
Let the server pace the polling, and push the news when it's done
A client that checks every second for a 200-second job sends 200 requests, almost all answered "still running". The server can slow it down: a Retry-After header, in seconds, says how long to wait before asking again, and status responses commonly carry it. Better still, the server can notify the client when the job finishes (Webhooks):
For one 200-second import. Shopify's bulk API sends a "bulk operation finished" webhook, and still lets clients poll.
Keep the status endpoint even when you send webhooks. A webhook can be lost or arrive late, a script may have no URL to receive one, and the status monitor is the one place that always has the truth. In a browser, the page that started the import can simply poll while it is open, and show progress from the job's metadata.
5 · Retries
Starting a job must be safe to retry
The request that starts a job can itself fail after the server accepted it: the 202 response is lost and the client tries again. Without care, the customer ends up with two imports of the same file and 200,000 duplicate rows. The client should send an ID it chose for this request, and the server should treat a repeat with the same ID and the same content as a retry, returning the same job (Idempotency). Microsoft's APIs use an Operation-Id header for this.
The jobs table is the status monitor's source. Workers can crash and the queue redelivers the job, so each step of the import must also be safe to repeat (Delivery semantics).
Large jobs need limits too. One customer starting 50 imports at once shouldn't take every worker: cap concurrent jobs per customer, as Shopify does with up to five bulk queries per shop at a time, and queue or reject the rest (Rate limiting algorithms).
6 · Bulk requests
A bulk request either succeeds as a whole or reports each item
Bulk APIs take many items in one call: create 1,000 subscribers, update 500 tags. Calling the single-item API 1,000 times wastes round trips; one bulk call doesn't. The design question is what happens when item 37 is invalid. There are two answers:
Aspect | All or nothing | Partial success |
|---|---|---|
Item 37 fails | Nothing is saved; the error names item 37 | The other 999 are saved; item 37 is reported |
Client then | Fixes item 37 and resends all 1,000 | Fixes and resends only item 37 |
Fits | Synchronous calls and small, related batches | Asynchronous jobs and big imports |
The usual rule | Synchronous batch calls must be atomic | Asynchronous ones may report failures by item index |
The subscriber import uses partial success, because nobody wants 199,412 good rows thrown away over 588 bad email addresses. The job's result reports the counts and links to a file listing each failed row and why, which the customer can fix and upload on its own. Set and document a maximum batch size; the larger the batch, the longer one call holds a server and the more there is to redo after a failure.
7 · Big results
Large results go to a file, not into a response
Exports have the same shape in reverse. A customer exporting 2 million subscribers through a paged API would need thousands of requests, each subject to rate limits, while the data changes underneath. Shopify's bulk operations show the alternative: submit one query, let it run in the background, and download the whole result from a signed URL as a JSON Lines file (one JSON object per line, so it can be read as a stream). The URL expires after one week.
The newsletter platform's export works the same way: the job writes the file to object storage and its result holds a short-lived signed link (Object & blob storage). The download never touches the API servers, and a failed download can simply be retried.
- Go async past a few seconds
Return 202 with a status monitor for anything that may outlast the timeouts on its path.
- Make the job a resource
Status, progress, result or error; keep it for at least a day after it ends; allow cancel.
- Make starting it idempotent
A client-chosen ID turns a retried start into the same job, not a second one.
- Pace and notify
Retry-After on status responses, a webhook on completion, and the status monitor as the source of truth.
- Report per item, deliver by file
Partial success with a list of failures for big batches; large results as a file behind a signed link.
Check yourself
4 questions
Takeaways
Remember this
- Work that may outlast the timeouts on its path should return 202 Accepted with a status monitor instead of holding the connection open.
- A job is a resource: status, progress, result or error, kept for a while after it ends, and cancelable.
- Starting a job must be idempotent, so a retried request finds the same job instead of starting a second.
- Let Retry-After pace polling, send a webhook on completion, and keep the status monitor as the source of truth.
- Big batches report failures per item; big results are delivered as a file behind a signed, expiring link.