System design interview: design a payment system

1. Clarify the requirements

Establish early that you will integrate a PSP rather than connect to card networks. Then agree on which flows matter.

• Functional: customers pay for an order by card or wallet; the business can capture later (when the order ships) and refund fully or partly; every payment, fee, refund and payout is recorded; payment status is visible to the order system and to finance.
• Non-functional: never charge twice and never lose a record of a charge; ledger writes are strongly consistent and auditable; an unknown outcome is shown as "processing" rather than guessed; a few seconds of latency is acceptable because the PSP call dominates.
• Out of scope unless asked: becoming an acquirer or card processor, building fraud models (assume the PSP's), currency conversion, and seller onboarding for marketplaces.

2. Estimate the scale

The assumed numbers mostly show that throughput is not the problem.

A well-indexed relational database handles 1,160 payments a second. Storage grows steadily, so the ledger is partitioned by time from the start and archived after the audit retention period.

3. Define the API

POST /v1/payments
  headers: Idempotency-Key: 3f9a61c2-order-5512-attempt-1
  body:    { "orderId": "ord_5512", "amount": 4999, "currency": "USD",
             "paymentMethod": "pm_tok_8Hq2", "capture": "later" }
  returns: 201 { "paymentId": "pay_77", "status": "AUTHORIZED" }
           same response again if the key was already used with the same body
           409 if a request with this key is still in progress
           422 if the key was reused with a different body

POST /v1/payments/pay_77/capture          (Idempotency-Key required)
POST /v1/payments/pay_77/refunds          (Idempotency-Key required)
  body:    { "amount": 1500 }

POST /webhooks/psp                        (from the PSP; signature verified)

pm_tok_8Hq2 is a token. The card number was typed into the PSP's hosted fields or SDK on the checkout page and sent directly to the PSP; your servers never see it. That tokenization keeps most of your backend out of PCI DSS scope.

4. Sketch the data model

Balances are derived from entries, never typed in. A worked example with an assumed processing fee of 1.75 on a 49.99 charge (any currency) shows why the PSP clearing account is so useful:

After the payout, PSP clearing nets to zero: 49.99 minus 1.75 minus 48.24. Any balance left in it after a settlement is money you expected and did not receive, or received and cannot explain, which is exactly what reconciliation looks for.

5. Charge exactly once on a network that delivers at least once

Exactly-once delivery over a network is not possible: a request can succeed and its response can be lost. What you can build is at-least-once attempts plus idempotent handling, so repeating a request has the same effect as sending it once.

1. 1Client creates the keyOne key per payment attempt, stored with the cart, so a double click or reload resends the same key.
2. 2Service claims the keyInsert the key as in_progress under a unique constraint, with the CREATED payment row, in one transaction. A completed key replays its response; an in-progress one gets 409.
3. 3PSP gets the same guaranteeCall the PSP with an idempotency key derived from payment_id. Stripe and Adyen both document idempotency keys on their APIs, so a retried call returns the original result instead of charging again.
4. 4Record the result atomicallyIn one database transaction: move the payment to its new state, append the ledger entries, write an outbox event, and store the response on the idempotency key.
5. 5Handle the unknownIf the PSP call times out, the payment stays PENDING. Retry with the same key, or look the payment up at the PSP; never create a new charge to find out.

Every transition is a conditional update, UPDATE payments SET status = 'CAPTURED', version = version + 1 WHERE payment_id = ? AND status = 'AUTHORIZED' AND version = ?. If zero rows change, another worker or webhook got there first, and nothing is written twice.

6. The architecture

7. Keep it consistent as it grows

• Strong consistency for the ledger. Ledger writes go to one primary inside transactions, replicated synchronously so a failover loses no committed entry: a deliberate choice of consistency over availability (see consistency and CAP). Reports can read lagging replicas. Multi-row transactions and constraints do real work here, which makes this a SQL rather than NoSQL workload.
• No distributed transactions. The PSP cannot join your database transaction, and two-phase commit across services is fragile. Use a saga: local transactions, each with a compensating action. Reserve stock, authorize, capture on shipment; if stock fails after authorization, void it; if the order is canceled after capture, refund.
• Outbox for events. A database write and a queue publish can succeed separately. Write the event to the outbox in the same transaction as the state change; a relay publishes it at least once, and consumers deduplicate by event_id. A receipt reaches the notification system this way.
• Webhooks. Verify the signature, store the event ID under a unique constraint, return 200 quickly, process asynchronously. Duplicates and out-of-order events are harmless with forward-only transitions. Poll the PSP for payments stuck in PENDING.
• Retries. Retry PSP calls only on network errors, 5xx and 429, with backoff and the same key. A decline is an answer, not an error.
• Hot accounts. Every payment credits the same revenue account, so a stored running balance there becomes a lock everyone waits for. Split it into sub-accounts summed on read, or compute it from entries in batches.
• Reconciliation. Daily, match every line of the PSP's settlement report to a payment by psp_reference (amount, fee, currency, status). Mismatches (missing on either side, amounts differ) go to an exceptions queue for finance, and the clearing account must net to zero for each payout.

8. What breaks, and what you do about it

9. Trade-offs to discuss

• Authorize then capture, or charge at once. Separate capture lets you take money only when goods ship and makes cancellation a free void, but authorizations expire after a period set by card network rules, so long delays need re-authorization.
• Synchronous or asynchronous confirmation. Waiting gives an immediate answer; confirming later tolerates a slow PSP. Many checkouts wait a few seconds, then show "processing".
• One PSP or several. A second PSP adds failover and routing options, but card tokens are usually tied to the PSP that created them, and reconciliation doubles.
• Your ledger or the PSP's dashboard as truth. The PSP knows only its own payments; your ledger also holds payouts, fees and manual adjustments.

10. How to talk through it in the interview

• Draw the state machine; it answers half of the follow-up questions.
• Use the words idempotency key, double-entry and reconciliation, and explain each in one sentence.
• Say why you avoided two-phase commit and what replaces it.

Practicing this design with Natively

Rehearse this as a mock interview: a friend asks the timeout and webhook questions on a call, or you read the prompt aloud and answer it alone. Natively transcribes the conversation live and can suggest talking points for each question as it comes in; comparing them with what you said is a quick way to spot gaps such as reconciliation or the outbox. You can capture a screenshot of your state machine sketch and get an explanation of it. The transcript and notes are kept for review afterwards, so you can check whether your ledger example actually balanced.