The same network call can arrive twice: a connection drops mid-response, the client retries on its own, a provider redelivers its webhook. Without a safeguard, the server creates two orders, sends two emails or charges twice. The idempotency key is the standard mechanism that recognises the repeat and returns the result of the first call, unchanged.
Why the same call arrives twice
HTTP semantics (RFC 9110) mark GET, PUT and DELETE as idempotent: replaying them leaves the server in the same state. POST is not, and it is precisely the verb that creates resources, triggers payments or pushes messages. A well-built client retries after a timeout without knowing whether the first request was processed before the connection failed.
The same uncertainty governs message queues and webhooks, which guarantee at-least-once delivery. A consumer therefore sometimes receives the same event twice, and a double click on an order button produces the same collision in the browser. What these cases share: the sender does not know whether the effect has already happened.
What an idempotency key does, and does not, guarantee
The principle fits in one sentence: the client attaches a unique identifier to each business operation, carried in the Idempotency-Key header. The server ties that key to the result it produced. If the key reappears, it returns the already-computed response instead of running the work again.
POST /v1/payments HTTP/1.1
Host: api.exemple.com
Idempotency-Key: 5f0c8b1e-2a3d-4c6f-9b21-7d0e4a9c1f88
Content-Type: application/json
{"amount": 4200, "currency": "eur", "order": "cmd_9182"}The scope of the mechanism is worth stating plainly. An idempotency key deduplicates two requests carrying the same key; it undoes nothing and does not reconcile two different keys aimed at the same intent. It turns a network “at least once” into an “exactly once” at the level of the effect, provided the key names the business intent rather than the HTTP attempt.
An idempotency key does not make the operation reversible: it only prevents it from happening twice.
The life cycle of an idempotent request
On arrival, the server looks up the key. Absent, it records it as “in progress”, runs the work, then stores the response code and body. Present and finished, it returns the stored response. Present but still running, it signals that an identical request is already in flight.
Storing keys: the heart of the mechanism
Everything rests on a dedicated table and a unique constraint. It is that constraint, not an application-level check beforehand, that arbitrates two concurrent requests: the insert that wins runs the work, the other fails and falls back to reading the stored result.
CREATE TABLE idempotency_keys (
id BIGSERIAL PRIMARY KEY,
scope TEXT NOT NULL, -- compte + endpoint
idem_key TEXT NOT NULL,
request_hash TEXT NOT NULL, -- empreinte du corps
status TEXT NOT NULL, -- 'in_progress' | 'completed'
response_code SMALLINT,
response_body JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
expires_at TIMESTAMPTZ NOT NULL,
UNIQUE (scope, idem_key) -- verrou atomique
);The handler is built around that atomic insert. It also compares a fingerprint of the body: the same key with a different payload signals abusive reuse, better rejected than served with a misleading result.
public function store(Request $request): JsonResponse
{
$key = $request->header('Idempotency-Key');
$scope = $request->user()->id . ':POST /v1/payments';
$hash = hash('sha256', $request->getContent());
// Insertion atomique : la contrainte d'unicité arbitre les concurrents
try {
DB::table('idempotency_keys')->insert([
'scope' => $scope, 'idem_key' => $key, 'request_hash' => $hash,
'status' => 'in_progress', 'expires_at' => now()->addDay(),
]);
} catch (QueryException $e) { // Cle deja presente : rejeu ou requete concurrente
$row = DB::table('idempotency_keys')
->where(compact('scope'))->where('idem_key', $key)->first();
if ($row->request_hash !== $hash) {
return response()->json(['error' => 'key_reuse'], 422); // Meme cle, corps different : reutilisation interdite
}
if ($row->status === 'in_progress') {
return response()->json(['error' => 'in_flight'], 409); // Operation encore en cours cote serveur
}
return response()->json($row->response_body, $row->response_code); // Rejeu : on renvoie la reponse d'origine, a l'identique
}
$payment = $this->charge($request); // Le traitement metier n'a lieu qu'une fois
DB::table('idempotency_keys')->where(compact('scope'))->where('idem_key', $key)
->update(['status' => 'completed', 'response_code' => 201,
'response_body' => $payment]);
return response()->json($payment, 201);
}The original response is kept as is, status code included, to be served again on replay.
{
"scope": "acct_42:POST /v1/payments",
"idem_key": "5f0c8b1e-2a3d-4c6f-9b21-7d0e4a9c1f88",
"request_hash": "b9f3...c1",
"status": "completed",
"response_code": 201,
"response_body": { "id": "pay_7Kd2", "amount": 4200, "status": "succeeded" },
"expires_at": "2026-08-30T22:00:00Z"
}Several deduplication strategies coexist depending on context.
| Approach | Guarantee | Cost | When to pick it |
|---|---|---|---|
| Database unique constraint | Atomic, resists concurrency | One index, one table | General case, transactional writes |
Application lock (Redis SET NX) | Fast, off the database | External dependency, expiry to manage | High traffic, response stored elsewhere |
| Deduplication by event id | Simple on the consumer side | Only covers streams with a stable id | Webhooks, message queues |
The traps in production
Concurrency. Two requests carrying the same key arrive at once. Without an atomic insert, both pass the “does the key exist?” check and run the work. The unique constraint settles the conflict at write time: the loser gets a 409 or waits for the in-flight response, never a second charge.
Three other pitfalls recur. The scope of the key must include the account and the endpoint: a key shared between two clients opens a response leak. Expiry keeps the table from growing without end; a window of twenty-four hours to a few days covers realistic retry windows. Finally, idempotency does not replace transactional consistency: the business write and the key-status update should share the same transaction, otherwise a response marked “completed” can outlive a charge that actually failed.
Client side and provider side: generating and consuming keys
The client generates a random key, a UUID v4 for instance, once per business intent. The decisive habit: keep that key for every retry of the same operation, and above all do not regenerate it on each HTTP attempt, or the guarantee vanishes. The major payment APIs expose the header under this exact name and return an explicit error on collision.
On the event-consumption side, the logic is symmetric. A webhook carries a stable identifier; the consumer keeps the identifiers already processed and ignores redeliveries. It is the same idempotency, applied to the input rather than the output.
What to take away
- An idempotency key turns an “at least once” delivery into an “exactly once” effect, as long as it names the business intent.
- Reliable deduplication rests on a unique constraint in the database, not on a prior application-level check.
- The key is scoped per account and per endpoint, expires after a short window, and the original response is served again unchanged.
- On the client, one key per operation, reused on every retry; on the consumer, deduplication by event id.
I’ve seen more double-billing incidents caused by a badly scoped key than by a missing one: a key reused for two distinct operations, or regenerated on every attempt, and the guarantee collapses silently. The rule I keep from the field comes down to two points: one key per business intent, never per HTTP request, and a test that deliberately replays the call before anything ships. — Simon Janvier
Further reading
Reference specification: The Idempotency-Key HTTP Header Field, IETF HTTP API working group.
