Designing idempotent payment integrations

At some point in the life of any payment integration, the same logical transaction arrives twice. A mobile client times out waiting for a response and retries. A load balancer routes a request, the upstream connection drops before the response returns, and the client's retry logic fires. A message queue redelivers because a consumer crashed before acknowledging. A human refreshes a page they shouldn't have. None of these are exotic edge cases — over a large enough transaction volume, they are a certainty, not a possibility. The question isn't whether duplicate requests will happen. It's whether the system was designed for that to be safe.

Idempotency is a property of the whole path, not a header

The common shorthand is "add an idempotency key" — a client-generated identifier sent with the request, checked against previously processed requests before doing anything else. That's necessary, but the key alone guarantees nothing if the rest of the path isn't built to honour it.

The check-then-act race. The most common way idempotency silently fails in practice: the handler looks up the key, doesn't find it, and proceeds to process the transaction — but "look up" and "record" are two separate operations, and two concurrent requests carrying the same key can both pass the lookup before either has written its record. The fix isn't "check more carefully" — it's making the record the atomic gate: a unique constraint on the idempotency key column, with the insert itself as the serialization point. If the insert conflicts, you already know the outcome — either return the original response, or if the first request is still in flight, return a definitive "in progress, do not retry yet" rather than silently processing a second time.

Idempotency has to reach the actual side effect, not just your own database. A payment integration frequently calls out — to a card network, a bank's core system, an ISO 8583-speaking switch, a QR facilitator. If your service recorded the idempotency key but crashed before the outbound call completed, and the retry re-enters your handler, does it re-issue the outbound call? If the downstream system doesn't itself support idempotent retries (its own de-duplication key, a network-assigned reference you can safely re-query by), you can end up idempotent from the client's perspective and non-idempotent at the point where money actually moves — which is the one place it matters most. ISO 8583 gives you the STAN and RRN for exactly this reason; a well-designed integration always checks "has this reference already been processed downstream" before re-sending, not just "have I seen this client-supplied key before."

State transitions need to be idempotent, not just creation. A transaction that has already settled and receives a duplicate "settle" message should recognize the no-op and return the existing result — not throw, not silently re-apply a balance update, and not treat a reversal retry as a fresh reversal. This means the state machine's transition function needs to be written as "given this event and the current state, what's the correct resulting state" rather than "apply this event's effect," with every transition checked against the current state before any mutation happens.

Responses have to be replayable. If a retry arrives for a request that already completed, the correct behaviour is returning the original response — same status, same reference, same amount — not re-deriving a fresh one that might legitimately differ (a fee that's since changed, a balance that's since moved for unrelated reasons). That means persisting the response payload alongside the idempotency record, not just a boolean "processed" flag.

Where this shows up architecturally

In practice this pushes a few decisions upstream of any individual endpoint:

• The idempotency key and its associated response live in the same transactional boundary as the state change they gate — not a separate cache that can drift out of sync with the source of truth.
• Downstream calls to card networks, banking cores or payment switches are wrapped with their own query-before-resend step, using whatever reference the downstream system itself considers canonical.
• Reconciliation exists as a standing process, not a manual escalation path — because even a correctly idempotent system can end up with a request that never got a response (the call succeeded downstream, but the response never made it back), and the only way to close that gap is comparing your own records against the downstream system's on a schedule, not waiting for a client to notice and complain.

None of this is exotic engineering — it's a small number of disciplined decisions, applied consistently, at exactly the points where a network can fail. The expensive version of this problem is discovering the gap after a customer has been double-charged; the cheap version is designing the gate before the first retry ever happens.