LLD Case Study: Payment Gateway

Advanced System Design & Software Architecture · Lesson 22 · Low-Level Design and Design Patterns

Lesson Overview

TechLearningPro's checkout already has a domain rule from Clean Architecture: a declined payment is not capturable, an approval code is minted by the gateway, and a timeout is UNCONFIRMED rather than a guessed approval. Hexagonal Architecture and Ports & Adapters put that call behind a port and kept it outside the database transaction. What those lessons left unspecified is the low-level design of the gateway module itself when there are two providers, when the learner's client retries, when a webhook arrives twice, and when an engineer is tempted to log "the card" to debug a decline.

This case study designs that module. The public command is "authorize this order for this amount with this payment-method token and this idempotency key." The module stores the outcome, captures up to the authorized amount, and refunds up to the captured amount. Providers sit behind PaymentProvider, an interface we own. A test double approves amounts at or below a configured threshold and is named as a double. A production adapter uses Spring's RestClient to call whatever path that acquirer documents, with the path and the host coming from configuration, not from a class that pretends to be a vendor SDK. This lesson does not invent a processor's REST resource or a Java method a vendor never shipped.

Card data stays outside our database. The API accepts a payment-method token that a provider-hosted field or a tokenizer already produced. There is no column for a primary account number and no column for a card verification value. Logs may contain the order id, the last four digits if a provider returned them as non-sensitive display data, and our idempotency key. They do not contain the token in full and they do not contain a PAN, because we never had one. That boundary is the PCI scope decision: our service is not a cardholder-data store.

Idempotency is a unique constraint on (principal_id, idempotency_key) plus a hash of the canonical request. The same key and the same hash return the stored result, including a stored decline. The same key and a different hash is a conflict, not a second charge. Retries run only when the outcome is unknown — a timeout or a 503 — and they send the same key. A decline is not retried. Spring's transaction covers the local row. It does not cover the provider HTTP call.

Learning Objectives

By the end of this lesson you will be able to:

1. Draw the payment states from create through authorize, capture, decline, unconfirmed, and refund, including which transitions are illegal.
2. Design an idempotency record that separates replay, conflict, and first attempt, scoped to the principal.
3. Place provider calls outside the database transaction and describe the crash window that remains.
4. Specify a retry policy that retries unknown outcomes with the same key and does not retry declines.
5. Keep PAN and CVV out of the domain, the schema, and the logs, and say what a tokenized request looks like.
6. Deduplicate webhooks by a provider event id after a signature check the adapter owns.
7. Test the use case with a threshold double and an in-memory store, and state what that double is not.

Prerequisites

• Lessons 5 and 6: CardGateway, declined versus unconfirmed, idempotency key, and the transaction that must not wrap HTTP. This lesson is the low-level design behind that port when the module grows capture, refund, a second provider, and webhooks.
• Lesson 17's State for the payment lifecycle and Strategy for provider selection. Lesson 18's unit of work and unique constraint. Lesson 19's composition root for choosing the provider adapter by profile.
• Java 21 and Spring Boot 3 with Spring Framework 6.1 RestClient and JdbcClient. JUnit 5. You do not need a merchant account. You do not need a vendor SDK on the classpath.

Detailed Core Concepts

The lifecycle is a state machine with money bounds

A payment is an aggregate. Its state:

• CREATED: the row exists, no provider outcome yet. Short-lived, inside the attempt.
• AUTHORIZED: the provider approved. authorizedAmount is set. capturedAmount is zero.
• DECLINED: the provider refused. No capture, no refund.
• UNCONFIRMED: we timed out or the connection failed after we may have sent the request. No capture. A retry with the same idempotency key may call the provider again.
• CAPTURED: capturedAmount is greater than zero and at most authorizedAmount. Further captures are allowed only up to the remainder if the product supports partial capture. This design allows partial capture until the remainder is zero, then the state stays CAPTURED.
• VOIDED: an authorization was cancelled before any capture. A void after capture is illegal; that is a refund.
• PARTIALLY_REFUNDED and REFUNDED: refunds are at most capturedAmount - refundedAmount.

Payment.capture(Money) throws if the state is not AUTHORIZED or a partially captured state we still allow, or if the amount exceeds the remainder. Payment.refund(Money) throws if nothing has been captured or the amount exceeds the refundable remainder. DECLINED and UNCONFIRMED refuse both. Those methods are the entity rules from lesson 5, extended with amounts. They do not call HTTP. The use case calls the provider first or the entity first depending on the operation, and the sequence section fixes the order.

Money is integer minor units and a currency. The currency is fixed at create. A capture in another currency throws. There is no floating point.

Idempotency keys are part of the aggregate's identity for the attempt

Clients send Idempotency-Key on authorize, capture, and refund. The key is unique per principal, not globally: two learners may choose the same UUID, and one must not read the other's receipt. Lesson 5 already required find(principalId, key). The row stores:

• principal_id, idempotency_key (unique together)
• request_hash of the canonical fields: operation, order id, amount, currency, payment-method token reference (a hash of the token, not the token), provider code
• payment_id and the serialized receipt once known
• state of the attempt: IN_PROGRESS, COMPLETED, CONFLICT is not stored — conflict is a comparison result

On arrival the store looks up the pair. No row: insert IN_PROGRESS with the hash, then do the work, then update to COMPLETED with the receipt. The insert is the lock. Two racing requests: one insert wins, the other gets a unique-constraint failure, reloads, and if the winner is still IN_PROGRESS returns a concurrent-in-progress result (HTTP 409 with a retry-after, or a wait — this design returns 409 IN_PROGRESS so the client retries the same key). If the winner is COMPLETED and the hash matches, return the receipt. If the hash differs, return 409 IDEMPOTENCY_CONFLICT and do not call the provider.

Hash mismatch is how a buggy client reuses a key for a different amount. Replaying the old receipt would hide the bug and might show the wrong money. Running the new amount under the old key can double-charge or charge a different order under a key the client thinks is safe. Conflict is the only honest result.

The provider is given the same idempotency key, or a key derived by a documented function of it, so the provider-side dedupe matches ours. Derivation is principalId + ":" + key if the provider's key space is global and principals must not collide there. The raw client key is not sent to the provider if it would leak into a shared tenant at the acquirer; the derived key is. The derivation is in the adapter. The domain stores our key.

Retries: unknown is retried, decline is not

Classify provider results as a sealed type the use case understands:

• Approved: approval code, provider reference, authorized amount.
• Declined: a reason code we define (CARD_DECLINED, INSUFFICIENT_FUNDS, DO_NOT_HONOR). The adapter maps vendor codes to these. The raw vendor paragraph is not stored.
• Unavailable: timeout, connection reset, HTTP 503, HTTP 429. We do not know whether the provider accepted the request.

Unavailable may be retried a small number of times with backoff, using the same derived idempotency key. Declined is returned to the client. Retrying a decline hammers the provider and can turn a single decline into a fraud signal. Approved is not retried.

Backoff belongs in the adapter or in a small RetryPolicy the adapter calls. It is not a loop inside Payment. A domain entity that sleeps couples a business rule to a latency policy. Spring Retry's @Retryable(retryFor = ProviderUnavailable.class, maxAttempts = 3, backoff = @Backoff(delay = 200, multiplier = 2)) is a real Spring Retry 2 annotation you may put on the adapter method. The lesson's reference implementation is an explicit loop in the adapter so the policy is visible without a proxy:

javaCopy
public ProviderDecision authorizeWithRetry(ProviderAuth request, RetryPolicy policy) {
    int attempt = 0;
    while (true) {
        attempt++;
        try {
            return client.authorize(request);
        } catch (ProviderUnavailable ex) {
            if (!policy.allow(attempt)) {
                throw ex;
            }
            policy.sleep(attempt);
        }
    }
}

policy.sleep in a test is a no-op counter. The production policy sleeps. RetryPolicy does not catch Declined. Declines are return values, not exceptions, so a broad catch (Exception) cannot "accidentally" retry them if the adapter author mapped declines to values. Connection failures throw ProviderUnavailable. That split is deliberate. If you map declines to exceptions too, retryFor must list only the unavailable type. A retry on RuntimeException will retry declines the day someone wraps them.

HTTP 408 and 429 and 503 are unavailable. HTTP 400 is a bug in our request: do not retry until the bug is fixed. HTTP 402 or the vendor's decline code is Declined. A timeout after the bytes may have been sent is Unavailable, not Declined, and not Approved.

The transaction boundary

Sequence for a new authorization:

1. Insert the idempotency row IN_PROGRESS in a short transaction. Commit. The unique constraint is now held by a durable row.
2. Call the provider outside any database transaction. Retry only Unavailable as above.
3. In a new short transaction, apply Approved or Declined to the payment, store the receipt, mark the idempotency row COMPLETED.

If the process dies during step 2, the row stays IN_PROGRESS. A client retry with the same key sees IN_PROGRESS and, if the row is older than a threshold, the use case may resume the provider call with the same key. If it is younger, return 409 IN_PROGRESS so two workers do not both call the provider. A reconciliation job, a driving adapter, scans stale IN_PROGRESS rows and asks the provider for the derived key's status if that provider documents a status query. If our adapter's provider has no status query, the job leaves the row for an attendant and never invents an approval. Lesson 5's rule: do not mint an approval code locally.

If step 3 fails after Approved, the provider has authorized and we have not recorded it. The idempotency row is still IN_PROGRESS if step 3 rolled back. Reconciliation or the client retry calls the provider with the same key. A provider that honors the key returns the original approval without a second hold. That is why the key is sent on the first attempt, not only on the retry. A provider that does not honor idempotency keys cannot be given blind retries. You then need their status query or you stop after a single timeout and reconcile manually. The adapter's capability interface makes this explicit: supportsIdempotentAuthorize() is a method on the adapter's metadata, or a separate port ProviderRecovery. Do not assume it. The threshold double supports it by remembering the key in a map.

Capture is the same shape: local IN_PROGRESS, provider capture outside the transaction, local update. Capture's idempotency key is distinct from authorize's key. Reusing the authorize key for capture is a hash conflict if the hash includes the operation, which it does. Good.

PCI boundary

PCI DSS treats the primary account number as cardholder data and treats the full track, CVV, and PIN as sensitive authentication data that must not be stored after authorization. The design consequence we implement is stronger than "encrypt the column": we do not accept the PAN at all.

The checkout page uses a provider-hosted field or a tokenizer. The browser posts the card to the provider. Our API receives paymentMethodToken, a string the provider minted, plus amount, currency, order id, and idempotency key. Payment has no pan field. The JPA or JDBC mapping has no such column. ArchUnit or a simple test can scan the schema migration for pan and cvv and fail. Logs use a redaction list: headers Authorization, body fields named paymentMethodToken, token, cardNumber. The last four and the brand may be stored if the provider returns them for display, because a receipt that says "Visa •••• 4242" is a product need. Those four digits are not a PAN. Do not log the full token; it is a bearer secret for charging that method.

Our service still has security obligations: authenticate the principal, bind the idempotency key to that principal, authorize that this principal may pay this order, and protect the provider's API secret in configuration, not in the repository. @ConfigurationProperties holds the base URL and the secret. The secret is not a domain field. Lesson 19's composition root builds the RestClient with the secret as a default header. Tests use the threshold double and never read a real secret.

This is not a PCI compliance certificate. It is the low-level design that keeps cardholder data out of the system so the scope conversation with an assessor starts from "we store tokens and outcomes," not from "we store cards and hope the disk is encrypted."

Webhooks

Some providers report capture or dispute asynchronously. The webhook adapter:

1. Reads the raw body. Signature verification needs the bytes the provider signed, not a re-serialized JSON object.
2. Verifies the signature with the provider's documented scheme, in the adapter. Failure returns 400 and writes nothing.
3. Parses a provider event id and a type.
4. Inserts the event id into provider_event with a unique constraint. A duplicate returns 200 and does not apply the transition again.
5. Maps the type to a domain command: RecordCapture, RecordRefund, RecordDispute. The command runs through the payment aggregate's methods, so a webhook cannot capture a declined payment by setting a column.

Unknown event types are stored as received and ignored by the state machine, with a metric. They are not a reason to return 500 forever; the provider would retry indefinitely. Return 200 after the unique insert, and let a human read unknown types. A 500 is reserved for a failure to persist the event id, so the provider will retry and we still dedupe later.

The domain does not verify HMAC. The adapter does. Swapping providers swaps the verifier. WebhookVerifier is a port implemented beside the RestClient adapter. The threshold double does not receive webhooks. A test calls the use case command the webhook would have called, and a separate test checks the verifier with a known signature fixture computed in the test from a test secret. Do not copy a vendor's production secret into the fixture.

Provider selection

ProviderDirectory maps a provider code to a PaymentProvider. The order or the merchant configuration chooses the code before the use case runs, and the command carries providerCode. The use case asks the directory. Adding a provider is a new adapter and a configuration entry, lesson 4's open-closed principle, without a switch in Payment. A switch (code) inside the use case is the design this directory exists to replace.

The interface we own:

javaCopy
public interface PaymentProvider {
    ProviderDecision authorize(ProviderAuth request);
    ProviderDecision capture(ProviderCapture request);
    ProviderDecision refund(ProviderRefund request);
}

ProviderAuth carries amount, currency, derived idempotency key, and payment-method token. It does not carry PAN. ProviderDecision is the sealed approved, declined, unavailable set. Capture and refund use the provider reference returned at approval, which we stored. They do not resend the token if the provider's capture API identifies the authorization by reference. The threshold double implements all three by an in-memory map.

RestClient appears only in an adapter class HttpPaymentProvider. The URI path is a property (provider.acme.authorize-path) because the path belongs to the acquirer's documentation, which this course does not reproduce and does not invent. The method calls are the real RestClient API: post(), uri(...), header("Idempotency-Key", derivedKey), body(...), retrieve(), and status handling. Error mapping lives in that class. If the property is blank, the bean fails at startup. The local profile does not create this bean. It creates ThresholdPaymentProvider.

Architecture and Design Diagrams

mermaidCopy
stateDiagram-v2
    [*] --> CREATED
    CREATED --> AUTHORIZED: approved
    CREATED --> DECLINED: declined
    CREATED --> UNCONFIRMED: unavailable after retries
    AUTHORIZED --> CAPTURED: capture
    AUTHORIZED --> VOIDED: void
    CAPTURED --> PARTIALLY_REFUNDED: refund part
    CAPTURED --> REFUNDED: refund all
    PARTIALLY_REFUNDED --> REFUNDED: refund remainder
    DECLINED --> [*]
    UNCONFIRMED --> AUTHORIZED: retry approved
    UNCONFIRMED --> DECLINED: retry declined

UNCONFIRMED is stored when retries are exhausted, not when the first packet fails, if the policy still has attempts left. Those attempts happen before the use case returns. The state diagram shows the durable state the client can observe.

mermaidCopy
sequenceDiagram
    participant Client
    participant Use as AuthorizePayment
    participant Store as PaymentStore
    participant Provider as PaymentProvider
    Client->>Use: command, Idempotency-Key
    Use->>Store: insert IN_PROGRESS or load
    alt Completed and hash matches
        Store-->>Use: receipt
        Use-->>Client: original receipt
    else Hash mismatch
        Use-->>Client: conflict
    else New or resumable
        Use->>Provider: authorize, derived key
        alt Approved or declined
            Provider-->>Use: decision
            Use->>Store: save payment and COMPLETED
        else Unavailable
            Provider-->>Use: unavailable
            Use->>Store: save UNCONFIRMED
        end
    end

The provider call sits between two short store transactions, not inside one. The diagram's "insert" commits before "authorize."

mermaidCopy
flowchart TB
    subgraph driving [Driving adapters]
        http[Checkout controller]
        hook[Webhook controller]
        job[Reconciliation job]
    end
    subgraph app [Application]
        auth[AuthorizePayment]
        cap[CapturePayment]
        ref[RefundPayment]
        dir[ProviderDirectory]
    end
    subgraph domain [Domain]
        payment[Payment]
    end
    subgraph driven [Driven adapters]
        jdbc[JdbcPaymentStore]
        threshold[ThresholdPaymentProvider]
        httpProv[HttpPaymentProvider]
        verify[WebhookVerifier]
    end
    http --> auth
    hook --> cap
    job --> auth
    auth --> payment
    auth --> dir
    dir --> threshold
    dir --> httpProv
    hook --> verify
    auth --> jdbc

ThresholdPaymentProvider and HttpPaymentProvider are alternative beans, selected by profile, not a chain that tries one and then the other. Falling through from a declined live provider to the threshold double would approve real declines in production the day the profile is wrong. One directory entry, one provider.

Technical Explanation

Aggregate transitions

javaCopy
public void capture(Money amount) {
    if (state != State.AUTHORIZED && state != State.CAPTURED) {
        throw new IllegalPaymentState(id, state, "capture");
    }
    if (!amount.currency().equals(currency)) {
        throw new CurrencyMismatch(id);
    }
    Money remainder = authorizedAmount.minus(capturedAmount);
    if (amount.isNegative() || amount.isZero() || amount.greaterThan(remainder)) {
        throw new AmountNotAllowed(id, amount);
    }
    capturedAmount = capturedAmount.plus(amount);
    state = State.CAPTURED;
}

A second partial capture uses CAPTURED with a remainder. When you want "single capture only," refuse capture unless capturedAmount is zero. This case study allows partial capture because course bundles sometimes ship in parts. The decision is on the entity, tested directly.

Refund:

javaCopy
public void refund(Money amount) {
    Money refundable = capturedAmount.minus(refundedAmount);
    if (refundable.isZero()) {
        throw new IllegalPaymentState(id, state, "refund");
    }
    if (amount.greaterThan(refundable)) {
        throw new AmountNotAllowed(id, amount);
    }
    refundedAmount = refundedAmount.plus(amount);
    state = refundedAmount.equals(capturedAmount) ? State.REFUNDED : State.PARTIALLY_REFUNDED;
}

Void requires AUTHORIZED and zero captured, sets VOIDED, and the use case calls provider void if the adapter implements it. If the provider has no void, the use case lets the authorization expire and records VOIDED locally only after the provider confirms expiry or void. Do not mark VOIDED locally while the hold remains. The command's result includes the provider reference.

Request hash

Canonical form is a single string with field separators the values cannot contain, or a length-prefixed encoding. This lesson uses a length-prefixed encoding so an amount cannot shift into the currency:

javaCopy
public static String hash(String operation, String orderId, Money amount, String tokenHash, String providerCode) {
    String canonical = operation + "\n" + orderId + "\n"
            + amount.minorUnits() + "\n" + amount.currency() + "\n"
            + tokenHash + "\n" + providerCode;
    return sha256Hex(canonical);
}

tokenHash is SHA-256 of the token, not the token. The idempotency row must not become a second copy of the bearer token. sha256Hex is JDK MessageDigest.getInstance("SHA-256"). The algorithm is fixed and stored implicitly by the column; if you ever change it, the column needs a version or old keys will look like conflicts.

Store operations with JdbcClient

The insert uses a unique constraint. DuplicateKeyException means another worker inserted. The adapter reloads. JdbcClient parameter binding is the same API as lesson 18. The payment update predicates on version so a webhook and a synchronous capture cannot lose each other's amount. Optimistic locking from lesson 18 applies to money. A conflict reloads and retries the domain method once if the transition is still legal, or returns 409. It does not add the capture amount twice.

HTTP adapter shape

javaCopy
public final class HttpPaymentProvider implements PaymentProvider {
    private final RestClient http;
    private final String authorizePath;

    public HttpPaymentProvider(RestClient http, String authorizePath) {
        if (authorizePath == null || authorizePath.isBlank()) {
            throw new IllegalArgumentException("authorizePath");
        }
        this.http = http;
        this.authorizePath = authorizePath;
    }

    @Override
    public ProviderDecision authorize(ProviderAuth request) {
        try {
            AcquirerAuthBody body = http.post()
                    .uri(authorizePath)
                    .header("Idempotency-Key", request.derivedKey())
                    .body(toBody(request))
                    .retrieve()
                    .onStatus(HttpStatusCode::is5xxServerError,
                            (req, res) -> { throw new ProviderUnavailable(res.getStatusCode().value()); })
                    .body(AcquirerAuthBody.class);
            return map(body);
        } catch (ResourceAccessException ex) {
            throw new ProviderUnavailable(ex);
        }
    }
}

RestClient, onStatus, and ResourceAccessException are Spring types. AcquirerAuthBody is our DTO for the JSON that acquirer returns, mapped in this class to Approved or Declined. The field names follow that acquirer's published schema in the project that has a contract. In this lesson the class is the boundary, and the threshold double is what tests run. Do not name this class after a real company unless you implement that company's published API and keep it updated. HttpPaymentProvider plus a configured path is the honest name. ResourceAccessException covers I/O failures from Spring's HTTP client. Map it to Unavailable. Do not map it to Declined.

Threshold double

javaCopy
public final class ThresholdPaymentProvider implements PaymentProvider {
    private final Money limit;
    private final Map<String, ProviderDecision> byKey = new HashMap<>();

    public ThresholdPaymentProvider(Money limit) {
        this.limit = limit;
    }

    @Override
    public synchronized ProviderDecision authorize(ProviderAuth request) {
        ProviderDecision existing = byKey.get(request.derivedKey());
        if (existing != null) {
            return existing;
        }
        ProviderDecision decision = request.amount().greaterThan(limit)
                ? new Declined("CARD_DECLINED")
                : new Approved("thr-" + request.derivedKey(), request.amount());
        byKey.put(request.derivedKey(), decision);
        return decision;
    }
}

The comment on the class states it is a test and local-profile double. Amounts above the limit decline. The same derived key returns the same decision, which is the idempotency behavior the retry test needs. It is not a processor. It does not open a socket. Profile local registers it. Production configuration does not.

Webhook dedupe

javaCopy
public void accept(String rawBody, String signatureHeader) {
    if (!verifier.verify(rawBody, signatureHeader)) {
        throw new InvalidWebhookSignature();
    }
    ProviderEvent event = parser.parse(rawBody);
    boolean inserted = events.insertIfAbsent(event.providerEventId(), event.type());
    if (!inserted) {
        return;
    }
    commandBus.apply(event);
}

There is no command bus framework. commandBus here is a badly chosen name in a snippet; the real type is WebhookApplication, a class that switches on the event type we support and calls CapturePayment or RefundPayment with an idempotency key equal to the provider event id. A duplicate event id returns before apply. Signature failure throws before the insert, so a forged event is not stored as a processed id that would block the real one. Order: verify, then insert, then apply. If apply fails, the transaction that inserted the event id rolls back, and the provider's retry can try again. That means apply and insert share one local transaction, and apply's provider call — if a webhook must call the provider again — is outside it. Many webhooks only record a fact the provider already settled. Those only touch our database and may sit inside one transaction. Do not call the provider for a new charge from a webhook handler that holds the event-id transaction open longer than the insert needs.

Practical Implementation or Design Walkthrough

Requirements

1. Authorize with a payment-method token, amount, currency, order id, provider code, principal, and idempotency key.
2. Same principal, key, and hash returns the stored receipt and does not call the provider again once COMPLETED.
3. Same key and a different hash returns conflict and does not call the provider.
4. Declines are stored as DECLINED and cannot be captured.
5. Timeouts become UNCONFIRMED after the retry policy is exhausted, with no approval code.
6. Capture amount is at most the authorized remainder. Refund amount is at most the captured remainder.
7. Provider HTTP runs outside the database transaction.
8. Webhook signature failures persist nothing. Duplicate event ids apply once.
9. No schema column and no log field for PAN or CVV. Tokens are not logged in full.

Non-functional: authorize's local transactions are two short ones, not one around HTTP. The provider adapter is replaceable. The threshold double is impossible to select in the production profile. Optimistic version on the payment row. Principal binding on every read.

Assumptions: one database; the provider either honors Idempotency-Key or is not configured for automatic retry (the adapter metadata says which); the client is authenticated before the controller builds the command; the order id has already been priced by the order module. This module does not recompute the catalog price. It charges the amount on the command and the order module's use case is responsible for passing the snapshotted price. A mismatch between order total and payment amount is checked by the order module before it calls authorize, or by this module if it is given the expected total. This design receives expectedOrderTotal and refuses if it differs from amount, so a client cannot pay a different total under a valid order id. The order module remains the owner of the price. The check is a guard.

Classes

409 for conflict, in-progress, and illegal state. 422 for a bad amount or currency. 402 is tempting for a decline and easy to confuse with a gateway error; this API returns 200 with a body state DECLINED or 402 consistently. Pick one and test it. This design returns 200 and DECLINED because the HTTP call succeeded and the business result is a decline. 502 is for UNCONFIRMED after retries if the client should retry the same key later; the body includes the payment id. Document it. A client that treats 502 as "send a new idempotency key" will double-charge if the first request actually approved. The response body and the API documentation say: reuse the key.

Tests

• Entity: capture on DECLINED throws; capture above authorized throws; refund above captured throws; currency mismatch throws.
• Idempotency: two calls, same key and hash, the double records one authorize. Different amount, same key, zero additional authorizes and a conflict.
• Race: two threads, one key, the in-memory store that enforces uniqueness, one provider call. The store must reject the second insert the way PostgreSQL would. A fake that ignores uniqueness is not this test.
• Retry: the double throws ProviderUnavailable twice and then approves for that key. The use case returns AUTHORIZED and the double shows three attempts and one stored approval. A decline on the first attempt shows one attempt.
• Transaction shape: a test provider that checks a TransactionSynchronizationManager.isActualTransactionActive() and fails the test if true. That Spring API is real. The use case test can run without Spring if the provider double simply counts calls; the synchronization check belongs in a Spring slice that runs the action. At least one test proves the provider is not called inside a transaction. If you cannot see an active transaction in the unit test because there is no Spring, assert the order of operations with a fake store that records "committed" before the provider method is allowed, by structuring PaymentStore.beginAttempt as a method the fake treats as complete before authorize returns.
• Webhook: bad signature does not insert; same event id twice captures once; declined payment ignores a capture event by throwing inside the aggregate and rolling back the event insert so a corrected event can retry — or records the event as rejected so a poison event does not retry forever. Poison versus transient needs a split: signature good, domain refusal, store the event id with outcome REJECTED and return 200 so the provider stops. Transient database failure rolls back and returns 500. Do not roll back forever on a domain refusal.

Trade-offs

• 200 with DECLINED versus HTTP 402. 200 keeps "transport worked" separate from "issuer said no" and is easier on clients that retry any 4xx with a new key. 402 matches some payment APIs and invites generic HTTP retry middleware to do the wrong thing. Prefer 200 plus a body if your gateway in front retries 4xx. Prefer a documented 402 if clients are yours and middleware is configured to retry only 502 and 503.
• Loop in the adapter versus @Retryable. The loop is visible and testable without a proxy. @Retryable is consistent if the whole codebase uses Spring Retry. Do not use both on the same method. Do not retry Exception.
• Insert idempotency row before the call versus after. Before the call, a crash leaves IN_PROGRESS, which is recoverable. After the call, a crash loses the approval and a retry might approve again if the provider key was not sent. Insert before, send the key always.
• Partial capture. Useful for split shipment, more states to test. Refuse it if every course purchase is one capture. The entity method is the place the decision lives.
• Status query versus blind retry. Blind retry is safe only with provider idempotency. Without it, stop and reconcile. Model the capability. Do not code as if every acquirer were the same.
• Storing last four. Good for support, a mild cardholder-data discussion if your assessor treats it strictly, and never a reason to store the rest. If you do not need the receipt line, store nothing but the brand.

Enterprise Real-World Scenarios

Two acquirers for a course marketplace

Domestic cards go to provider north. A second region uses provider south because of settlement currency. ProviderDirectory holds both adapters, each a RestClient with its own base URL, secret, and path properties. The merchant row on the order carries providerCode. The use case does not contain if (region). A decline mapping table per adapter translates that acquirer's codes into CARD_DECLINED and INSUFFICIENT_FUNDS. Unmapped codes become CARD_DECLINED plus a log of the unmapped code, not the PAN. The log line includes provider code and vendor code. Support can ask the acquirer. The domain stays on our enum.

The threshold double is not in the directory in production. A misconfigured profile that loads it would approve every amount under the threshold with approval codes thr-*. A startup check fails the production profile if any registered provider's class is the threshold type. That check is a @Bean method in the production configuration, ten lines, worth more than a comment.

Launch day retries: the policy is three attempts, 200 ms, 400 ms, only on ProviderUnavailable. A 503 storm retries inside the request and holds the servlet thread. If the storm continues, the policy's max attempts return UNCONFIRMED quickly and the client retries later with the same key. Do not raise attempts to hide an outage. The reconciliation job, one active instance like the elevator controller, scans UNCONFIRMED and IN_PROGRESS older than two minutes. It uses the status query when the adapter has one. It never sets AUTHORIZED without a provider reference.

Webhook-driven refunds and a support console

A learner's refund can start from our API or from the provider's dashboard. Dashboard refunds arrive as webhooks. The webhook path calls Payment.refund with the amount in the event. A duplicate delivery is the unique event id. A refund event for an unknown provider reference is stored as REJECTED and alerted; it does not create a payment. Creating payments from webhooks would let a forged-but-signed event — signing secrets leak — invent charges. Signature verification assumes the secret is still secret. The allow-list of event types is short: refund and dispute. Charge events that we already applied synchronously are stored and ignored when the payment is already CAPTURED for that amount. If the amounts differ, raise a conflict for a human. Do not "fix" the row to the webhook's amount without a domain transition that conserves money.

The support console is a driving adapter that sends RefundPayment with an idempotency key the console generated and stored. It does not update SQL. A console that set state = REFUNDED skips the provider call and the bounds check. The next webhook then refunds at the provider and the aggregate throws, or the provider was never asked and the learner was not refunded while the UI says they were. All money movement goes through the use case.

PCI shows up in the console too. The console displays last four and brand and the provider reference. It has no screen that accepts a card number "for manual charge." Manual charges use a provider virtual terminal outside this service, then a webhook or a typed provider reference that an attendant command reconciles. Typing a PAN into our admin form would undo the boundary. The form does not have the field.

Best Practices

• Accept tokens, never PAN or CVV. Do not add the columns "just in case." Redact tokens in logs.
• Bind idempotency keys to the principal. Hash the canonical request. Conflict on hash mismatch. Send a derived key to the provider on the first attempt.
• Insert IN_PROGRESS and commit before the HTTP call. Complete the receipt in a second transaction.
• Retry only unavailable outcomes, with a capped policy, using the same key. Do not retry declines or HTTP 400.
• Keep provider calls off the transaction. Prove it with a test.
• Map vendor decline codes at the adapter. Store our codes.
• Verify webhook signatures on raw bytes. Dedupe on provider event id. Domain refusals return 200 after recording REJECTED. Database failures return 500 so the provider retries.
• Use the aggregate for capture and refund bounds. Do not set amounts in SQL from a controller.
• Select providers through a directory. Keep the threshold double off the production profile with a startup check.
• Version the payment row. Money updates are compare-and-set.
• Document client retry: same key on 502 and UNCONFIRMED, new key only for a new operation.
• Separate authorize, capture, and refund idempotency keys. Include the operation in the hash.

Common Mistakes and Anti-Patterns

Storing the card "encrypted, temporarily." Temporary becomes the backup. The API that accepts PAN puts you in scope for cardholder data even if you delete it quickly. Take a token. Do not add the endpoint.

Retrying every exception. A decline wrapped in RuntimeException is retried three times and the issuer flags the card. Retry only ProviderUnavailable. Declines are values.

One transaction around RestClient. The pool exhausts, and a rollback after a successful authorize does not void the hold. Two short transactions and a key.

Global idempotency key. Learner B reuses learner A's key, by collision or by a leaked receipt, and receives A's result. The unique constraint includes principal_id. The lookup does too.

Same key, different amount, return the old receipt. The client shows the new amount and you charged the old one, or the reverse. Return conflict.

Webhook applies UPDATE payment SET state = CAPTURED. Declines get captured because the SQL skipped Payment.capture. Call the entity.

Re-serializing JSON before verifying the signature. Field order changes and every real webhook fails, or you verify a different document than the provider signed. Verify the raw body.

Logging the token and the Authorization header. The log store becomes the card-data store. Redact at the encoder.

Threshold double on the production classpath as @Primary. Every "declined" large payment under the limit approves. Profile it and assert the production directory's types.

Inventing a vendor client. A class named for a real processor with methods you guessed will be wired to production by someone who trusts the name. Call it HttpPaymentProvider until it implements a published contract you can point at.

Reconciliation that sets AUTHORIZED when the provider cannot be reached. That mints approval codes. Leave UNCONFIRMED.

Troubleshooting and Design Review

Symptom: duplicate holds at the acquirer. Retries used a new idempotency key, or the first request never sent a key, or two principals were omitted and two charges shared a provider key by accident. Check the derived key is stable for our key. Check the client library does not generate a new key on 502.

Symptom: legitimate retry returns conflict. The hash includes a field the client correctly changes, such as a timestamp you should not have included. The canonical set is operation, order id, amount, currency, token hash, provider code. Remove volatile fields. A timestamp in the hash makes every retry a conflict.

Symptom: IN_PROGRESS rows older than the request timeout. The process died between insert and complete, or the provider hung past the client timeout and the thread is still in RestClient. Set the client's read timeout. Let reconciliation resume with the same key after a staleness threshold. Do not start a second charge under a new key automatically.

Symptom: capture succeeds locally and the provider later says the authorization expired. You captured after the auth window. Store the provider's expiry if they send one, and refuse capture after it, calling void or re-authorize with a new key as a new operation. A new operation is a new idempotency key. Do not reuse the authorize key for a second authorize of a higher amount.

Symptom: webhook storms of 500. A domain exception rolls back the event id, so every retry fails again. Record REJECTED and return 200 for illegal state. Return 500 only when the insert itself fails.

Symptom: declined payment becomes captured. A webhook or a console updated the column. Search for SQL that sets state. The only writer should be the repository saving an aggregate that has already transitioned.

Symptom: PCI scope review finds card numbers in logs. A debug logger printed the HTTP body. Turn off body logging for the provider client, or use a logger that applies the redaction list. Add a test that a sample body containing a PAN-like field is redacted. Even though our API rejects PAN, the provider response might echo more than you asked. Map responses through a DTO that has only the fields you intend to keep.

Design review. Are PAN and CVV absent from types, schema, and logs? Does hash mismatch refuse the call? Is the provider outside the transaction? Does retry exclude declines? Are webhook signatures checked on raw bytes before insert? Does a duplicate event id no-op? Can DECLINED be captured in a unit test? Is the threshold double excluded from production? Does reconciliation refuse to invent approvals? Is the idempotency key principal-scoped? Is partial refund bounded by captured minus refunded?

Interview Questions and Answers

Q1. What do you store, and what do you refuse to store, for a card payment?

You store a payment-method token reference or its hash, amount, currency, state, provider reference, approval code, idempotency key, principal, and optionally brand and last four for display. You refuse the PAN, the CVV, the full track, and the PIN. The checkout sends the card to a tokenizer or a hosted field. Our endpoint does not have those parameters. Encryption of a PAN column is not the design; absence is.

Q2. How does idempotency behave for the same key and a different amount?

The unique key is (principal_id, idempotency_key). The row stores a hash of the canonical request, including the amount. A replay with the same hash returns the stored receipt and does not call the provider again after completion. A replay with a different hash returns a conflict and does not call the provider. Returning the old receipt would lie about the amount. Executing the new amount under the old key can double-charge.

Q3. Why is the provider call outside the database transaction?

The transaction holds a connection and rolls back only our rows. It does not roll back a hold the acquirer already accepted. Holding it during HTTP exhausts the pool when the acquirer slows. The design commits an IN_PROGRESS idempotency row, calls the provider, then commits the outcome. The crash between approval and the second commit is handled by the same key on retry and by reconciliation, not by a bigger transaction.

Q4. Which failures do you retry, and which do you surface immediately?

Retry timeouts, connection failures, HTTP 503, and HTTP 429, using the same derived idempotency key, up to a small attempt limit. Do not retry declines, HTTP 400, or hash conflicts. Declines are Declined values so a catch of ProviderUnavailable cannot include them. After the limit, persist UNCONFIRMED without an approval code.

Q5. A webhook and a synchronous capture race. How do you keep the captured amount correct?

The payment row has a version. Both paths load, call Payment.capture or the equivalent recording method, and update where version = ?. One commits. The other gets OrderConflict's cousin, PaymentConflict, reloads, and applies only if the remainder still allows it. The webhook is deduped by event id so a replay does not add the amount twice even if the version moved for another reason. Domain methods add amounts. SQL does not add a literal from the controller.

Q6. How do you stop a second process from double-authorizing the same key?

The first transaction inserts the idempotency row. The unique constraint allows one winner. The loser reloads. If the state is IN_PROGRESS and fresh, it returns in-progress and does not call the provider. If the state is COMPLETED, it returns the receipt. Reconciliation is single-active or uses the same row as a lease by updating IN_PROGRESS to a claimed timestamp with a conditional update. Two reconcilers cannot both be the claimer.

Q7. Why is a class named for a real card processor a design smell here?

Unless it implements that processor's published API and you maintain that contract, the name is a false dependency. Call the adapter HttpPaymentProvider, put the path in configuration, and map JSON to our ProviderDecision in one place. Tests use ThresholdPaymentProvider, which approves under a limit and remembers keys, and is not selectable in production.

Q8. What is the difference between DECLINED and UNCONFIRMED for a later capture?

DECLINED means the provider refused. Capture is illegal. UNCONFIRMED means we do not know. Capture is also illegal until a retry or a status query returns Approved and the aggregate moves to AUTHORIZED. Treating unknown as approved writes an approval code we invented. Treating unknown as declined may hide a hold that actually succeeded and will expire or settle. Keep the third state.

Q9. How should the client retry a 502 from authorize?

With the same idempotency key and the same body. A new key is a new operation and can create a second hold if the first one succeeded. The API documentation and the error body say so. Middleware that generates a fresh key per attempt is a defect. Hash conflicts on a changed body are the signal that the retry was not a retry.

Q10. Where does signature verification live, and what do you persist if the signature is wrong?

In the webhook adapter, over the raw body, before parse side effects and before the event-id insert. A bad signature persists nothing and returns 400. If you insert first, an attacker who can write your database can fill event ids, or you store garbage you must not apply. Verification uses the provider's scheme and a secret from configuration. The domain aggregate never sees the header.

Q11. How does this module stay consistent with lessons 5 and 6?

The use cases depend on PaymentStore and PaymentProvider ports. Payment does not import Spring or RestClient. The HTTP adapter and JdbcClient store are driven adapters. The controller, the webhook, and the reconciliation job are driving adapters. @Transactional is on the short store methods, not around the provider. The threshold double is the same idea as lesson 5's gateway double, extended with capture, refund, and key memory.

Q12. What do you do with a provider that cannot honor idempotency keys?

You do not blind-retry authorize. One attempt, then UNCONFIRMED on timeout, and an attendant or a status query if they have one. You may still idempotent-protect our own API so our client retries do not start a second attempt while the first is IN_PROGRESS. The adapter exposes that it lacks provider-side dedupe, and the retry policy reads that flag. Pretending every provider matches the one you tested first is how the second acquirer double-charges.

Hands-On Exercises

Exercise 1: Idempotent authorize

Problem. Implement Payment, the idempotency store contract, and AuthorizePayment using ThresholdPaymentProvider.

Tasks.

1. Specify hash fields. Exclude timestamps. Include operation and amount.
2. Same key and hash returns one provider call after completion. Different amount conflicts with zero extra calls.
3. A decline cannot be captured. Prove it on the entity without Spring.
4. ProviderUnavailable retries up to three times with the same derived key; a decline does not.
5. Describe the two database transactions around the call. The provider method must not require an open transaction.
6. Confirm the type has no PAN or CVV field.

Rubric. Requirements including PCI boundary 15%. State and idempotency design 25%. Tests for replay, conflict, decline, and retry 20%. Transaction and crash window 20%. Trade-offs 10%. Clarity 10%.

Exercise 2: Webhook refund

Problem. Accept a refund webhook: verify a test signature, dedupe on event id, apply Payment.refund for a captured payment.

Tasks.

1. Bad signature inserts nothing.
2. Two deliveries with one event id refund once.
3. A refund larger than the captured remainder records REJECTED and does not move the amount. Define the HTTP status you return and why the provider will not retry forever.
4. A concurrent capture uses a version column in the store you describe, even if the in-memory fake only simulates it.
5. The handler does not call a provider to take a new charge.

Rubric. Same weights. A handler that sets state in a map without Payment.refund does not pass the design criterion.

Lesson Summary

The payment module is a state machine with money bounds, an idempotency row scoped to the principal and hashed to the request, and a provider port that sees tokens and derived keys. Declines and unknowns are different states. Retries apply only to unknowns, with the same key, and only when the provider's idempotency story supports them. Database transactions bookend the HTTP call and do not wrap it. Webhooks verify raw signatures, dedupe on event id, and enter through the same aggregate methods as the API. PAN and CVV are not in the model. The threshold double proves the use case and is barred from production. The directory isolates a second acquirer. Lessons 5 and 6 supplied the dependency rule; lessons 17 through 19 supplied state, repository, unit of work, and injection; this case study is those rules applied to a charge that must happen once and to a card number that must not be stored at all.

Further Reading

• PCI DSS documentation from the PCI Security Standards Council, for cardholder data and sensitive authentication data. Use it as scope guidance. This lesson is not an assessor’s report.
• Spring Framework reference: RestClient, JdbcClient, transaction management, and TransactionSynchronizationManager for the "no transaction during HTTP" test.
• Spring Retry reference: @Retryable and @Backoff, if you place retries on the adapter. Retry the unavailable type only.
• Clean Architecture and Hexagonal Architecture and Ports & Adapters for the port and the transaction split this case study implements.
• Lesson 18 for the unique constraint, version column, and unit of work around the local rows.
• IETF HTTP Idempotency-Key header draft, for the header convention. Principal scope and request hashing are our contract on top of that header.