Domain Events

Introduction

Domain events record something meaningful that happened in the domain — past tense, immutable facts consumed by other aggregates and contexts. Netflix billing, entitlement, and playback contexts coordinate via events like SubscriptionActivated and ProfileMigrated without synchronous coupling.

Real production story

Netflix migrated profiles between account tiers using synchronous REST calls from billing to playback — a billing deploy timeout caused playback entitlement checks to fail globally, stopping streams for 40 minutes. The remediation introduced domain events: Billing emits SubscriptionTierChanged; playback maintains local entitlement projection updated async with idempotent consumer; sync API retained only for user-initiated immediate upgrade with SLA. Playback decoupled from billing deploy cadence; similar incidents eliminated; entitlement lag bounded to 30 seconds with explicit member messaging during propagation.

Business problem

Netflix subdomains (signup, billing, playback, recommendations) evolve independently. Synchronous coupling for state propagation creates correlated outages and blocks team velocity.

• Member experience: Stream should not stop because billing service redeployed.
• Scale: Millions of tier changes during promotions — sync fan-out does not scale.
• Audit: Regulators and fraud teams need immutable history of entitlement changes.

Architecture overview

Domain event naming: past tense domain verb — SubscriptionActivated, not UpdateSubscription. Payload: aggregate ID, occurredAt, minimal data, schema version.

• vs integration event: Domain event is inside context; ACL may translate to public integration event.
• Raising: Aggregate collects events; repository persists aggregate + outbox in one transaction.
• Handling: One handler per reaction; idempotent on eventId; no domain logic in handler glue only.
• Ordering: Per aggregate ID partition key preserves order for that stream.

Architecture motivation

Domain events decouple contexts while preserving causal history. Raised from aggregates after state change; persisted with outbox; consumed with idempotent handlers.

• Force: Cross-context reactions (billing → playback → email) without transaction.
• Constraint: Eventual consistency acceptable with bounded lag and UX handling.
• Outcome: Event catalog per context; schema registry; replay for new projections.

Internal architecture

Netflix entitlement event flow — billing to playback decoupling:

• Playback never calls billing HTTP on start-stream hot path — reads local projection.
• Event version v2 parallel consume during schema migration.

Billing Context
  Subscription aggregate
     │ activate()
     ▼
  SubscriptionActivated { subId, tier, memberId, at, v:2 }
     │
     ▼
  Outbox → relay → Kafka topic billing.subscription.v2
     │
     ├─► Playback: EntitlementProjector (idempotent)
     ├─► Email: WelcomeTierEmailHandler
     └─► Analytics: (read-only, dropped on lag OK)

Playback Context
  local EntitlementCache (eventually consistent, TTL 30s)

Data flow

Transactional outbox: aggregate save + insert outbox row same DB transaction → relay polls outbox → publish to broker → consumer updates projection → mark processed with eventId dedup table.

• Upgrade path: Member clicks upgrade → sync API for immediate path + event for all consumers.
• Replay: New fraud service replays billing events from offset 0 to build projection.
• Failure: Consumer lag alert — playback serves stale tier max 30s then blocks with clear UX.

System design diagram

Two diagrams show the Domain Events topology and the primary request/event path used in production at scale.

Production code example

Aggregate + outbox pattern — TypeScript domain event wiring:

• pullDomainEvents clears buffer after persist — prevents double publish on retry.
• Outbox insert same transaction as aggregate — atomic reliability boundary.

export class Subscription extends AggregateRoot {
  activate(tier: Tier): void {
    if (this.status !== SubscriptionStatus.PENDING) {
      throw new InvalidTransitionError(this.status, "activate");
    }
    this.status = SubscriptionStatus.ACTIVE;
    this.tier = tier;
    this.raise(new SubscriptionActivated({
      eventId: EventId.generate(),
      subscriptionId: this.id,
      memberId: this.memberId,
      tier: tier.code,
      occurredAt: new Date().toISOString(),
      schemaVersion: 2,
    }));
  }
}

export async function saveSubscription(
  repo: SubscriptionRepository,
  outbox: Outbox,
  sub: Subscription
): Promise<void> {
  await repo.transaction(async (tx) => {
    await tx.save(sub);
    for (const evt of sub.pullDomainEvents()) {
      await outbox.insert(tx, {
        aggregateId: sub.id.value,
        eventType: evt.constructor.name,
        payload: JSON.stringify(evt),
        eventId: evt.eventId,
      });
    }
  });
}

Enterprise case study

Netflix billing-playback decoupling: Sync entitlement API caused global stream outage on billing deploy.

• Before: Playback sync call billing on every start-stream; 40-min global outage on billing deploy.
• Decision: Domain events + local entitlement projection + outbox + 30s staleness SLA.
• After: Zero playback outages from billing deploys in 24 months; tier propagation p99 < 8s.

Trade-offs

• Eventual consistency vs UX: Immediate sync for user-facing actions; events for propagation — hybrid is OK.
• Event granularity: Fine events verbose; coarse events hard to evolve — balance per consumer needs.
• Choreography vs orchestration: Events enable choreography; complex sagas may need orchestrator.
• Storage: Event log retention cost — tier hot/cold; not every event kept forever unless event sourcing.

Security considerations

Events carry minimum data: SubscriptionActivated includes tier enum and IDs — not full payment instrument.

• Encryption: Kafka TLS + ACL per consumer group; no PII in topic name.
• AuthZ: Consumers verify event signature or source mTLS — prevent forged entitlement events.
• Retention: Shorter retention on sensitive topics; audit log separate long-retention store.

Scalability analysis

Netflix event volume requires partition strategy, schema evolution, and consumer autoscale independent of producer.

• Partition key: memberId or subscriptionId — order per member stream.
• Schema registry: Avro/Protobuf with compatibility BACKWARD — consumers on old schema during deploy.
• Fan-out: Dedicated consumer groups per downstream — playback lag does not block email.

Failure scenarios

Event failure modes: lost events without outbox; duplicate processing without idempotency; schema break; poison message infinite retry.

• Dual write: DB commit without outbox — relay never fires; use transactional outbox always.
• Duplicate handler: At-least-once delivery — store processed eventIds.
• Poison: DLQ after N tries; alert; fix forward with new schema version.

Staff engineer insights

• Domain events are facts, not commands — if name sounds imperative, you're probably doing RPC with extra steps.
• Outbox is non-optional for money and entitlement paths — 'we'll add events later' means you'll duplicate production traffic wrongly first.
• Every consumer team should answer 'what happens if we're 6 hours behind?' — if answer is Sev-1, fix projection or UX degradation.

Interview questions

Architecture review questions

• Are events past tense domain facts with schema version?
• Is transactional outbox used for all domain event publishes?
• Are consumers idempotent on eventId?
• Is partition key chosen for ordering requirements?
• Is max consumer lag documented with UX degradation plan?
• Is event payload minimal and free of cross-context entity graphs?

Summary

Domain events at Netflix decouple billing from playback with past-tense facts, transactional outbox, and idempotent projections — eliminating global stream outages caused by synchronous entitlement coupling.