Event Sourcing

Introduction

Event sourcing persists state as an append-only sequence of domain events rather than mutable rows. Current state is derived by replaying events — giving Stripe-grade audit trails, temporal queries, and natural integration via event streams. It pairs often with CQRS but stands alone when audit and reconstructability are primary forces.

Real production story

Stripe's ledger team initially stored account balances as updatable rows. A bug in a deployment double-applied a refund adjustment — balances were wrong but logs showed only final numbers. Forensics could not answer "what did we think the balance was at 14:32 UTC?" for regulators and card networks.

The team moved to event-sourced ledger entries: every charge, capture, refund, and dispute is an immutable event. Balance is a projection; disputes replay history to prove correctness. PCI and SOC audits consume the event stream directly; rollbacks become compensating events, not silent overwrites.

Business problem

Business pressure: Stripe processes trillions in payment volume — regulators, partners, and merchants demand immutable financial history and explainable balance changes.

• Audit / compliance: Mutable rows fail "show your work" requests during disputes.
• Debugging: Production balance mismatches need time-travel diagnosis, not log archaeology.
• Integration: Downstream fraud, reporting, and ML want every state transition, not snapshots.

Architecture overview

Event store appends events keyed by aggregate ID with optimistic concurrency (expected version). Projections fold events into read models; snapshots truncate replay for hot aggregates.

• Definition: State = fold(events); mutations = new events only.
• When to adopt: Audit, temporal queries, complex lifecycles (orders, accounts, workflows).
• When to defer: Simple CRUD with no audit burden — event store ops cost is real.
• Operability: Version events; upcasters migrate old schemas on replay.

Architecture motivation

Why architects care: Event sourcing makes history the source of truth — snapshots become optimization, not authority.

• Force: Financial and identity domains require append-only audit semantics.
• Constraint: Replay time must stay bounded — snapshot strategy mandatory at scale.
• Outcome: New projections (reporting, analytics) without migrating core tables.

Internal architecture

Stripe-style ledger event store:

POST /v1/charges  →  ChargeService
                              ↓ append
                    EventStore (Kafka / dedicated ES DB)
                    stream: account-{id}
                    events: [ChargeCreated, CaptureSucceeded, ...]
                              ↓
              ┌───────────────┼───────────────┐
              ↓               ↓               ↓
      BalanceProjector   ReportingETL    FraudScoring
              ↓
      BalanceSnapshot (every N events or daily)
              ↓
      GET /balance  reads projection + tail replay

Data flow

All mutations append; deletes are tombstone events.

• Write path: Load aggregate from snapshot + events since; validate; append new event with version check.
• Read path: Query projection or replay aggregate in memory for strong consistency.
• Async path: Event bus fans out to consumers; each maintains idempotent cursor.

System design diagram

Two diagrams show the Event Sourcing topology and the primary request/event path used in production at scale.

Production code example

Aggregate append with optimistic concurrency:

class AccountAggregate {
  apply(event: AccountEvent): void { /* fold state */ }
}

async function handleRefund(cmd: RefundCommand): Promise<void> {
  const stream = await eventStore.loadStream(cmd.accountId);
  const account = stream.fold(AccountAggregate.empty());
  if (account.balance < cmd.amount) throw new InsufficientFunds();

  const event: RefundIssued = {
    type: "RefundIssued",
    accountId: cmd.accountId,
    amount: cmd.amount,
    chargeId: cmd.chargeId,
    occurredAt: new Date().toISOString(),
  };

  await eventStore.append({
    streamId: cmd.accountId,
    expectedVersion: stream.version,
    events: [event],
  });

  await outbox.publish(event);
}

Enterprise case study

Stripe ledger event sourcing — immutable payment lifecycle for audit and dispute resolution.

• Before: Mutable balance rows; incident forensics averaged 6 engineer-days.
• Decision: Append-only ledger events + balance projections + periodic snapshots.
• After: Dispute replay in minutes; SOC2 evidence exported from event stream.

Trade-offs

• Audit vs complexity: Event schema evolution, upcasting, and snapshot management are permanent tax.
• Storage growth: Append-only streams grow forever — compaction and archival policies required.
• Query ergonomics: Ad-hoc SQL reporting needs projections; event store is not a data warehouse.
• Learning curve: Teams accustomed to ORM updates struggle with "no UPDATE" mental model.

Security considerations

Immutable audit trail is also immutable breach exposure if events contain secrets.

• PII in events: Store references (token IDs) not PAN/email in payload — GDPR erasure uses tombstones.
• Integrity: Signed events or hash chains for tamper-evidence in regulated ledgers.
• Access: Event store read access highly restricted — equivalent to production DB admin.

Scalability analysis

Hot aggregates (popular merchant accounts) accumulate huge streams — snapshots critical.

• Horizontal scale: Partition event streams by aggregateId; no cross-partition transactions.
• Replay cost: Snapshot every 500 events or hourly; parallel rebuild workers for new projections.
• Cost: Cold-tier archival for events older than 7 years — regulatory retention vs cloud bill.

Failure scenarios

Concurrency conflicts and poison replays dominate incident history.

• Version conflict: Two writers append same version — retry with refreshed aggregate state.
• Upcaster bug: Old events mis-parse after deploy — canary replay on shadow projection first.
• Projection drift: Consumer skipped event — rebuild from checkpoint; never patch projection rows manually.

Staff engineer insights

• Event sourcing is not "Kafka everywhere" — you still need aggregate boundaries and invariants on command side.
• Snapshot strategy is day-one architecture, not performance tuning later — hot aggregates will hurt.
• Upcasters are forever code — budget maintenance like API versioning.

Interview questions

Architecture review questions

• Events are immutable — corrections via compensating events only.
• Optimistic concurrency on append — expectedVersion enforced.
• Snapshot and upcaster strategy documented with replay SLA target.
• No secrets in event payloads — token references only.
• Projection rebuild runbook tested — no manual SQL fixes on read models.
• Retention and archival policy meets regulatory and cost requirements.

Summary

Event sourcing gives Stripe-level auditability by storing every state change as an immutable event. Aggregates replay history; projections and snapshots keep reads fast — at the cost of schema evolution discipline and operational maturity.