EventBridge, SNS and SQS

AWS Cloud & DevOps Engineering · Lesson 22 · AWS Application Platform

Lesson Overview

An enrollment does not finish when the HTTP response returns 200. Payment confirmation, seat reservation, progress board unlock, email receipt, and analytics all need to happen—and they must not share one brittle call stack that times out together. Synchronous APIs create the order. Asynchronous messaging makes the Monday launch survive the blast of “I enrolled” clicks without turning every downstream outage into a learner-facing failure.

This lesson is where TechLearningPro stops treating “fire and forget” as an accident and starts treating events, topics, and queues as first-class platform contracts. AWS Lambda and Serverless Fundamentals gave you compute that scales with invocations. API Gateway and Serverless APIs gave you the edge that accepts the order. Here you decide how that acceptance becomes reliable work for many consumers—without inventing a second monolith that polls a status column forever.

You will learn Amazon EventBridge for routing typed events across buses and rules, Amazon SNS for fan-out pub/sub, and Amazon SQS for durable buffering with visibility timeouts, long polling, and dead-letter queues. You will design for at-least-once delivery and idempotent consumers. You will know when FIFO ordering helps and when it becomes a throughput bottleneck. Cleanup and budget discipline still apply: every topic, queue, and custom bus you create for a lab should leave with you.

Next lesson—Step Functions and Workflow Orchestration—turns these events into explicit multi-step sagas. Later, Project 3: Serverless Event-Driven Order Platform will assemble the full stack. This lesson builds the nervous system those workflows and projects depend on.

Learning Objectives

By the end of this lesson you will be able to:

1. Explain why TechLearningPro order and enrollment paths need asynchronous messaging instead of only chained synchronous HTTP.
2. Describe EventBridge event buses, rules, event patterns, and targets at an implementation level.
3. Choose between the default bus, a custom application bus, and partner/SaaS-oriented patterns without over-sharing events.
4. Design SNS topics for fan-out notifications and multi-subscriber progress updates.
5. Use SQS standard queues to buffer work for Lambda or other consumers with correct visibility timeout thinking.
6. Configure long polling and explain how short polling burns empty receives.
7. Implement fan-out with SNS→SQS and EventBridge→multiple targets, and defend when each pattern fits.
8. Attach and operate dead-letter queues (DLQs) for SQS, SNS, and EventBridge failure paths.
9. Design idempotent consumers that survive at-least-once delivery without double-charging or double-enrolling.
10. Compare standard versus FIFO queues: ordering guarantees, throughput implications, and when FIFO is the wrong default.
11. Wire retries so transient failures recover and poison messages leave the hot path via DLQ.
12. Apply least-privilege IAM and resource policies for publishers and consumers, including secrets-adjacent configuration from Secrets Manager and Parameter Store.
13. Tag, budget, alarm, and destroy lab messaging resources so queues and topics do not become silent spend.
14. Sketch the order-processing event flow that Project 3 will harden into a production-shaped platform.

Prerequisites

• AWS Lambda and Serverless Fundamentals: handlers, execution roles, concurrency, and event source mapping intuition.
• API Gateway and Serverless APIs: how the order or enrollment request enters the platform.
• AWS Identity and Access Management: roles, resource policies, and least privilege.
• Secrets Manager and Systems Manager Parameter Store: no webhook secrets or API keys in message bodies or topic policies by accident.
• A sandbox account with a budget alert before creating queues, topics, buses, or Lambda consumers.
• Comfort with JSON event payloads and the idea that delivery is not the same as successful business processing.

Detailed Core Concepts

The problem messaging solves for TechLearningPro

When a learner completes checkout, the platform must: persist the order, reserve a seat in a cohort, charge or confirm payment, unlock course access, emit a progress update, notify the learner, and feed analytics. Doing all of that inside one API Lambda forces every dependency’s latency and failure mode into the learner’s waiting browser. A payment provider blip becomes a 502. An email outage blocks enrollment.

Asynchronous messaging separates acceptance from fulfilment. The API writes durable state and publishes an event or enqueues work. Downstream consumers process at their own rate. Backpressure appears as queue depth, not as cascading timeouts. That is the reliability shape for enrollment week—not a hope that every microservice stays healthy at the same millisecond.

Three tools, three jobs

Treat the trio as complementary, not interchangeable:

EventBridge is a router with pattern matching. SNS is a broadcast channel. SQS is a buffer with receipt handles and visibility. Teams that force every problem into one service invent awkward workarounds—filtering in every consumer, or polling SNS as if it were a queue.

Amazon EventBridge: buses, rules, targets

An event bus receives events. AWS services emit to the default bus. Your applications should usually publish domain events to a custom bus (tlp-orders, tlp-progress) so application traffic is not mixed with every AWS service event in the account, and so IAM can scope who may put events on which bus.

A rule matches events using an event pattern (source, detail-type, detail fields) or a schedule. Matched events go to one or more targets: Lambda, SQS, SNS, Step Functions, API destinations, and other supported integrations documented by AWS. Rules evaluate independently; one event can match many rules.

Event shape for application events typically follows the EventBridge envelope: source, detail-type, detail, plus metadata AWS adds. For TechLearningPro, prefer stable contracts:

jsonCopy
{
  "source": "tlp.orders",
  "detail-type": "OrderPlaced",
  "detail": {
    "orderId": "ord_01HZX…",
    "learnerId": "lrn_…",
    "courseId": "course-aws-devops",
    "amountCents": 49900,
    "currency": "USD"
  }
}

Version the contract deliberately (detail-type or a schemaVersion inside detail). Breaking consumers silently is how Monday becomes an incident.

Archive and replay (where enabled for your bus design) support recovery after a bad consumer deploy. Do not treat replay as a substitute for idempotency—replay will redeliver.

Schema Registry awareness: documenting and optionally registering schemas helps producers and consumers stay aligned. Teaching depth here: treat schemas as product contracts owned by a team, not as console afterthoughts.

Amazon SNS: topics and subscriptions

An SNS topic is a named channel. Publishers send a message to the topic; subscriptions deliver copies to endpoints. Fan-out is the product: one publish, many delivery attempts to independent subscribers.

For platform work, prefer application topics with SQS or Lambda subscriptions over email-to-engineer topics for production fulfilment. Email and SMS subscriptions are valuable for human alerts; they are poor substitutes for durable processing pipelines.

Message attributes and filter policies let subscriptions receive only relevant messages (for example, only courseId prefixes for a partner). Filtering at SNS reduces wasted Lambda invocations—but filters must be tested; a wrong filter looks like “SNS is broken” when it is correctly silent.

SNS FIFO topics exist for ordered, deduplicated pub/sub into FIFO queues. Use when ordering across fan-out matters and you accept FIFO constraints. Most TechLearningPro notification fan-out is fine on standard topics.

Amazon SQS: queues as buffers

An SQS queue stores messages until a consumer receives and deletes them. Producers send; consumers long-poll, process, then delete. If processing fails before delete, the message becomes visible again after the visibility timeout—that is the retry mechanism baked into the queue.

Standard queues offer nearly unlimited throughput, at-least-once delivery, and best-effort ordering. FIFO queues offer exactly-once processing within the deduplication window when configured correctly, and ordered processing per message group—at lower throughput ceilings. Default to standard unless a business invariant requires FIFO.

Visibility timeout must exceed worst-case successful processing time (including downstream calls). Too short: another consumer picks up a still-running message → duplicates under load. Too long: failed messages wait forever before retry. Align with Lambda function timeout when using event source mapping; AWS documents the relationship—set both consciously.

Long polling (ReceiveMessageWaitTimeSeconds up to 20) reduces empty responses. Short polling in a tight loop burns API calls and cost for nothing. For Lambda event source mappings, AWS manages polling; for custom pollers, enable long polling.

Message retention (up to the documented maximum, commonly thought of in days) defines how long unprocessed work survives. Short retention with a slow consumer equals silent data loss. Prefer alerting on age-of-oldest-message before retention expires.

Delay queues and per-message delay postpone visibility for backoff-style designs. Do not confuse delay with a scheduler product; for complex schedules prefer EventBridge Scheduler or Step Functions Wait states (next lesson).

Fan-out patterns

SNS → multiple SQS queues. Classic durable fan-out: one business event, separate queues for email, analytics, and seat reservation so each consumer scales and fails independently. Subscribe each queue to the topic; grant SNS permission to send to each queue via queue policy.

EventBridge → multiple targets. One rule with several targets, or several rules with different patterns. Prefer EventBridge when routing depends on content (detail-type, fields) across many producers, or when you want bus-level isolation and archive/replay. Prefer SNS when you want a simple pub/sub topic as a product API for many subscriber teams.

EventBridge → SNS or SQS. Valid composition: EventBridge routes; SNS fans out further; SQS buffers a specific worker. Avoid infinite conceptual stacks—“EventBridge to SNS to EventBridge to SNS”—without a clear ownership reason.

Direct Lambda as target vs SQS in front of Lambda. Direct targets are simpler. SQS in front provides a buffer when producers outpace consumers, when you need DLQ semantics tied to receive counts, or when you want to smooth bursts. For spiky enrollment, SQS in front of Lambda is often the kinder design to concurrency limits.

Dead-letter queues and failure paths

A dead-letter queue receives messages that could not be processed successfully after a configured number of receives (SQS redrive policy), or that failed delivery along SNS/EventBridge paths according to each service’s DLQ support.

DLQs are not trash cans you ignore. They are quarantine with an owner. Alarm on DLQ depth. Runbook: inspect payload, classify poison vs transient systemic failure, fix consumer or payload, replay carefully with idempotency, then delete or archive.

For SQS→Lambda, understand the difference between function errors that cause retries and messages that eventually redrive to DLQ. For SNS, delivery failures to a subscription can use a DLQ on the subscription where supported. For EventBridge, failed invocations to targets can use a DLQ on the target configuration—verify current console/API options in docs when you implement; do not invent fields.

At-least-once delivery and idempotency

SQS standard and many fan-out paths deliver at least once. Network retries and visibility expiry create duplicates. Your consumer must treat “process OrderPlaced for ord_01HZX” as safe to run twice: use an idempotency key (order ID), conditional writes, or an processed-events store.

Idempotency is a business property, not an SQS checkbox. FIFO deduplication reduces duplicate publishes within a window; it does not remove the need for careful consumer design across all failure modes (for example, process succeeds, delete fails, message returns).

TechLearningPro anti-pattern: charging a card on every message receive. Correct pattern: charge once keyed by orderId / payment intent ID; subsequent receives observe “already charged” and complete successfully (ack/delete).

Ordering: when FIFO helps and when it hurts

FIFO helps when a single aggregate must be processed in order—for example, progress updates for one learnerId+courseId where out-of-order “lesson 5 complete” before “lesson 4 complete” corrupts a state machine. Use message group IDs so different learners progress in parallel while one learner’s stream stays ordered.

FIFO hurts when you put the entire platform on one message group (global ordering): throughput collapses to a single-threaded pipe. It also hurts when the business does not need order—enrollment confirmation emails do not need FIFO. Prefer standard queues plus idempotent, commutative updates when possible.

EventBridge and SNS standard paths do not give you a global order promise. Design consumers that do not require one unless you deliberately chose FIFO topologies.

Security and encryption awareness

Encrypt queues and topics with AWS-owned or customer managed KMS keys per your account standard (later KMS deep dive expands this). Resource policies on queues must allow only intended principals (SNS service principal for subscriptions, specific roles for send/receive). Do not put secrets, raw card data, or long-lived tokens in message bodies—reference IDs and fetch secrets at process time via IAM.

Cross-account subscribe/publish requires explicit resource policies. “Temporary * for debugging” on a queue policy is how another account’s workload becomes your consumer.

Observability minimum

Alarm on: queue depth, age of oldest message, DLQ depth, EventBridge failed invocations, SNS delivery failure metrics where available. Correlate with Lambda errors and throttles. A silent DLQ is an outage with a delay. Module 4 deepens CloudWatch; this lesson requires you not to ship messaging without depth and age alarms.

Cost drivers without fake prices

You pay for API requests (publish, receive, delete), payload size tiers, and related data transfer as documented. Chatty short polling and huge fan-out to unused subscribers waste money. Idle queues cost little compared to forgotten Lambda concurrency storms from recursive triggers—but unused custom buses and unused subscriptions still add operational noise. Delete lab resources. Prefer long polling. Filter SNS subscriptions. Do not invent unit prices in design docs; link current pricing pages when finance asks.

Architecture and Design Diagrams

Order acceptance to fan-out fulfilment:

mermaidCopy
flowchart LR
  API["API Gateway + Order Lambda"]
  EB["EventBridge custom bus\ntlp-orders"]
  R1["Rule: OrderPlaced"]
  SF["Step Functions\n(next lesson)"]
  SNS["SNS topic\nenrollment-events"]
  Q1["SQS: seat-reservation"]
  Q2["SQS: notify-learner"]
  Q3["SQS: analytics"]
  DLQ["SQS DLQ"]
  API -->|"PutEvents"| EB
  EB --> R1
  R1 --> SF
  R1 --> SNS
  SNS --> Q1
  SNS --> Q2
  SNS --> Q3
  Q1 -.->|"after maxReceive"| DLQ
  Q2 -.-> DLQ

SNS→SQS fan-out with independent consumers:

mermaidCopy
flowchart TB
  Pub["Publisher: enrollment service"]
  Topic["SNS: EnrollmentConfirmed"]
  EmailQ["SQS email-worker"]
  PushQ["SQS push-worker"]
  PartnerQ["SQS partner-webhook"]
  Pub --> Topic
  Topic --> EmailQ
  Topic --> PushQ
  Topic --> PartnerQ

Visibility timeout and retry loop (conceptual):

textCopy
SendMessage → message invisible to others while consumer works
  → success: DeleteMessage
  → timeout/crash before delete: message visible again → retry
  → receiveCount ≥ maxReceiveCount → redrive to DLQ

EventBridge content routing:

mermaidCopy
flowchart TB
  Producers["orders / payments / progress producers"]
  Bus["Custom bus tlp-domain"]
  RuleA["detail-type = OrderPlaced"]
  RuleB["detail-type = PaymentFailed"]
  RuleC["source = tlp.progress"]
  Producers --> Bus
  Bus --> RuleA
  Bus --> RuleB
  Bus --> RuleC
  RuleA --> TargetsA["Lambda + SQS + workflow"]
  RuleB --> TargetsB["notify + compensate path"]
  RuleC --> TargetsC["board updater queue"]

Technical Explanation

Publishing without coupling

Producers should depend on a bus name or topic ARN, not on the list of consumers. New analytics subscriber? Add a subscription or rule—do not change the order Lambda. That inversion is the entire point of pub/sub and event routing. Document the event contract in a shared place; code review producers that invent one-off field names per ticket.

Event patterns that stay maintainable

Match on source and detail-type first. Add detail field matchers sparingly. Over-specific patterns break when optional fields are absent. Under-specific patterns invoke the wrong targets. Test patterns with sample events in the console or CLI before production cutover. Prefer additive evolution: new optional fields, new detail-types for new meanings—not silent reuse of OrderPlaced for “order updated.”

SQS event source mapping to Lambda

Lambda polls SQS (AWS-managed) and invokes your function with a batch. Partial batch failure reporting (where you use the supported response shape) lets successful messages delete while failures retry—verify the current Lambda SQS partial failure behavior in docs when implementing. Set batch size and maximum concurrency so you do not overwhelm DynamoDB or a payment API. A queue is a buffer; unbounded Lambda concurrency can empty the buffer into a thundering herd on a dependency.

Receipt handles and delete semantics

Consumers must delete using the receipt handle from the receive, not the message ID alone. Extending visibility (ChangeMessageVisibility) helps long-running work; for Lambda, prefer fitting work into the function timeout or moving long work to Step Functions (next lesson) instead of heroic visibility extensions in ad-hoc pollers.

Poison messages

A message that always fails (schema break, missing course ID, permanent 400 from a dependency) must not block a FIFO group or burn retries forever. maxReceiveCount + DLQ isolates poison. On standard queues, poison still burns retries and metrics—alarm and fix. On FIFO, a poison message can stall a message group until handled—another reason not to use one global group.

Choosing EventBridge vs SNS vs SQS (decision cues)

• Need content-based routing across many event types and producers → EventBridge.
• Need simple fan-out of the same message to many subscribers → SNS.
• Need buffer, retry, and competing consumers → SQS.
• Need ordered per aggregate → FIFO queue (and possibly FIFO SNS) with thoughtful group IDs.
• Need scheduled emission → EventBridge schedules (or Scheduler), not a home-grown sleep loop in Lambda.

Integration with secrets and config

Webhook signing secrets for partner HTTPS subscriptions, third-party email API keys used by workers, and bus ARMs belong in Secrets Manager or Parameter Store—not in message attributes. The worker role reads the secret; the message carries partnerId and orderId only.

Testing strategy

Unit-test idempotency with duplicate payloads. Integration-test: publish once, assert each queue received; force a consumer exception, assert retry then DLQ; publish an event that should not match a rule, assert no invocation. In labs, use distinct name prefixes and tags (Project=tlp-lesson22) so cleanup is grep-friendly.

Performance and backpressure

Queue depth rising means consumers are slower than producers—scale consumers, reduce batch work, or shed load intentionally. Age of oldest message rising toward retention is a data-loss countdown. SNS fan-out to a slow HTTP endpoint should use SQS buffering rather than blocking other subscribers. EventBridge rule limits and quotas exist—check Service Quotas for your account rather than assuming infinite rules.

Multi-account sketch

Platform account owns the bus; domain accounts put events via resource policy or event bus-to-bus routing. Do not open PutEvents to * . Organization-level event strategies appear in later multi-account lessons; here, know that buses and policies are the seam.

Practical Implementation or Design Walkthrough

Problem

Design and implement a minimal order-processing path in a sandbox: API or CLI producer publishes OrderPlaced; EventBridge routes to an SNS topic; SNS fans out to two SQS queues (fulfilment and notifications); a Lambda consumer on fulfilment demonstrates idempotency and failure→DLQ behavior. Budget first; clean up everything.

Steps

1. Confirm budget alert. Tag all resources Project=tlp-lesson22, Owner=<you>.
2. Create two standard SQS queues: tlp-order-fulfilment and tlp-order-notify, plus tlp-order-fulfilment-dlq. Attach redrive on fulfilment with a small maxReceiveCount (for example 3) pointing at the DLQ.
3. Set visibility timeout on fulfilment greater than the Lambda timeout you will use (for example Lambda 10s, visibility 30–60s—tune deliberately; read current AWS guidance for the pair).
4. Create SNS topic tlp-enrollment-events. Subscribe both queues; update each queue policy to allow sns:SendMessage from that topic ARN.
5. Create custom EventBridge bus tlp-orders. Create a rule matching source=tlp.orders, detail-type=OrderPlaced, target = SNS topic (and optionally a CloudWatch Logs target for lab visibility if appropriate in your account posture).
6. Create a Lambda fulfilment consumer with an execution role limited to receive/delete on the fulfilment queue, write metrics/logs, and—if you simulate payments—no broad IAM. Implement idempotency keyed by orderId (for lab: DynamoDB conditional write or a simple table; if you skip DynamoDB, document how you would key idempotency and simulate with an in-memory set only for single-instance demos—prefer DynamoDB for honesty).
7. Add SQS event source mapping from fulfilment queue to Lambda.
8. Publish a test event with PutEvents onto tlp-orders. Confirm messages appear on both queues; fulfilment Lambda succeeds; notify queue can be drained manually or by a second function.
9. Publish a poison payload that fails validation every time. Confirm retries and DLQ receipt. Alarm or at least note DLQ depth.
10. Publish the same valid orderId twice. Confirm idempotent handling (second success without double side effects).
11. Cleanup: delete event source mappings, Lambdas, rules, bus, subscriptions, topics, queues (including DLQ), lab tables, log groups if your policy requires, and verify tagged resources are gone.

Illustrative event put (adapt account Region and bus name; do not invent unsupported fields):

bashCopy
aws events put-events --entries '[
  {
    "Source": "tlp.orders",
    "DetailType": "OrderPlaced",
    "EventBusName": "tlp-orders",
    "Detail": "{\"orderId\":\"ord_lab_001\",\"learnerId\":\"lrn_1\",\"courseId\":\"aws-devops\"}"
  }
]'

Illustrative consumer sketch (conceptual):

pythonCopy
# Pseudocode — validate against current Lambda/SQS batch response docs when coding
def handler(event, context):
    for record in event["Records"]:
        body = parse_sns_or_sqs_envelope(record)
        order_id = body["orderId"]
        if not already_processed(order_id):
            reserve_seat(body)  # must be idempotent or transactional with marker
            mark_processed(order_id)
        # delete is handled by Lambda on success for the record

Failure paths to rehearse

• Remove queue permission for SNS → subscription fails delivery; observe metrics and fix policy.
• Set visibility shorter than work duration → duplicate invocations; fix timeout pairing.
• Omit idempotency → double seat reservation on retry; then add the marker and retest.

Enterprise Real-World Scenarios

Enrollment week without a buffer

TechLearningPro emailed a new cohort launch. The order API invoked email, CRM, and analytics synchronously. Email provider latency spiked; API timeouts surged; learners refreshed and double-submitted. Repair: accept order, publish OrderPlaced, SNS→SQS per consumer, idempotent fulfilment on orderId. API latency collapsed to persist+publish. Email lag became queue depth, not checkout failure.

The shared default bus

A team published application events to the default EventBridge bus alongside AWS service events. Rules proliferated; one overly broad pattern invoked a costly Lambda on irrelevant AWS events. Repair: custom bus for tlp.* sources; default bus reserved for AWS and carefully scoped rules; IAM denies PutEvents to default for application roles.

FIFO as a global lock

Progress updates used a FIFO queue with a single message group ID progress for the whole site. Throughput stalled during exams. Repair: message group ID = learnerId#courseId; accept per-learner ordering only; switch notification traffic back to standard queues.

DLQ without an owner

Fulfilment DLQ grew for nine days. Nobody paged. Finance asked why refunds spiked—duplicate manual fixes after silent loss of automation. Repair: alarm on DLQ depth > 0 for 5 minutes to the order squad; runbook for replay; weekly DLQ burn-down as an SLO input.

Partner fan-out blast radius

One SNS topic delivered to an external HTTPS partner. Partner outage caused SNS retries; other subscribers were fine, but noisy failure metrics hid a second bug. Repair: partner gets its own SQS subscription and worker with isolated DLQ; topic remains fan-out; failure isolation becomes tangible.

Cross-team contract break

Payments renamed amount to amountCents without a new detail-type. Analytics broke; fulfilment coerced nulls. Repair: schema ownership, consumer contract tests on sample events, additive versioning, and dead-letter on validation failure instead of guessing.

Best Practices

• Publish domain events to a custom EventBridge bus with stable source and detail-type names.
• Prefer SNS→SQS for durable multi-consumer fan-out; use EventBridge when content routing is the main need.
• Design every consumer for at-least-once delivery with an explicit idempotency key.
• Set visibility timeout and Lambda timeout as a pair; rehearse under failure.
• Attach DLQs, alarm on depth and age, and assign an owning squad.
• Use long polling for any custom receiver; avoid empty short-poll loops.
• Default to standard queues; use FIFO only with per-aggregate message group IDs.
• Grant least-privilege IAM on send/receive/subscribe; tighten queue and topic policies.
• Keep PII minimization in payloads; store sensitive tokens in Secrets Manager, not messages.
• Tag Project, Owner, Environment; budget before labs; delete mappings before functions before queues.
• Version event contracts; add consumers without editing producers.
• Filter SNS subscriptions intentionally; test both match and non-match cases.
• Prefer buffering with SQS in front of Lambda for bursty enrollment over unbounded direct fan-in.
• Document replay procedure before you need archive/replay in anger.
• Cross-link workflows: when fulfilment becomes multi-step with compensation, move orchestration to Step Functions and keep queues/events as triggers and side effects.

Common Mistakes and Anti-Patterns

Synchronous chain of Lambdas for enrollment side effects. Symptom: timeouts and double-clicks. Cause: no buffer. Fix: events/queues and idempotency.

Ignoring DLQ. Symptom: silent loss after retries. Cause: no alarm/owner. Fix: DLQ + page + runbook.

Visibility timeout shorter than processing. Symptom: duplicate side effects. Cause: timeout mismatch. Fix: align timeouts; make idempotent.

FIFO with one message group for all traffic. Symptom: latency cliff. Cause: global ordering. Fix: per-aggregate groups or standard queue.

Broad EventBridge patterns on the default bus. Symptom: surprise invocations and cost. Cause: weak filters / wrong bus. Fix: custom bus + tight patterns.

Secrets in message bodies. Symptom: leakage via logs and DLQ dumps. Cause: convenience. Fix: references + Secrets Manager.

Short polling loops in ECS/EC2 workers. Symptom: high SQS request cost, empty receives. Cause: WaitTimeSeconds = 0. Fix: long poll.

Deleting queues before event source mappings or subscriptions. Symptom: stuck lab cleanup or orphan mappings. Cause: wrong teardown order. Fix: mapping → function → rule/subscription → topic → queue.

Assuming SNS delivery equals business success. Symptom: “SNS succeeded” but learner not enrolled. Cause: confusing transport ACK with domain commit. Fix: metrics on consumer success and DLQ.

No idempotency because “FIFO exactly-once.” Symptom: duplicates after process-success/delete-failure edge cases. Cause: misunderstanding guarantees. Fix: still key side effects.

Troubleshooting and Design Review

Messages not arriving on SQS from SNS. Check subscription status, queue policy for SNS principal and topic ARN, KMS key policy if SSE-KMS blocks the service, and filter policies that exclude your attributes.

EventBridge rule never fires. Confirm bus name on PutEvents, rule enabled, pattern vs actual source/detail-type, and target permissions (EventBridge needs rights to invoke targets—role or resource policy per target type).

Lambda not draining queue. Event source mapping state, function errors, throttles, reserved concurrency zeroed elsewhere, or IAM missing sqs:ReceiveMessage/DeleteMessage/GetQueueAttributes.

DLQ filling. Read sample messages; classify validation vs dependency outage; fix consumer or pause producer; replay with care.

High cost / API noise. Short polling, excessive fan-out, recursive publish loops (consumer republishes same event without guard).

Duplicates in fulfilment. Visibility too short, missing idempotency, or multiple competing consumers with non-idempotent writes.

Design review checklist:

• What is the idempotency key for enrollment/payment side effects?
• Which bus/topic is the contract, and who owns schema changes?
• What is maxReceiveCount and who owns the DLQ at 02:00?
• Standard vs FIFO—what ordering invariant exists?
• What alarms fire before retention drops a message?
• Can a new consumer onboard without changing the producer?
• Lab cleanup evidence?

Interview Questions and Answers

Q1. Why introduce SQS between SNS and Lambda for enrollment notifications?

SQS provides a durable buffer, independent scaling, visibility-based retries, and a clear DLQ path so a slow or failing consumer does not drop messages at the topic and so bursts do not translate directly into Lambda concurrency spikes.

Q2. What is the difference between EventBridge and SNS?

EventBridge routes events using content patterns across buses and many AWS targets; SNS is pub/sub fan-out from a topic to subscriptions. Use EventBridge for typed enterprise event routing; SNS for straightforward broadcast; they can compose.

Q3. Explain visibility timeout.

After a consumer receives a message, it stays hidden from other consumers for the visibility timeout. If not deleted before then, it reappears for retry. It must fit successful processing time.

Q4. How do you handle at-least-once delivery in payment fulfilment?

Make the charge and enrollment writes idempotent on a business key (orderId / payment intent). Retries and duplicates become safe no-ops after the first successful commit.

Q5. When would you choose a FIFO queue?

When you need ordered processing per message group and deduplication for a specific aggregate (for example one learner’s progress stream), and you accept throughput limits. Not as a default for all traffic.

Q6. What belongs in a DLQ runbook?

Alarm threshold, owner, how to inspect payloads safely (PII), classify poison vs outage, fix forward, replay procedure with idempotency, and when to drop messages with approval.

Q7. Why prefer a custom EventBridge bus for TechLearningPro domain events?

Isolation from AWS service traffic on the default bus, clearer IAM boundaries, and cleaner governance of application event contracts.

Q8. How does long polling help?

Receive calls wait for messages to arrive (up to the configured wait), cutting empty responses and wasted API volume versus tight short polls.

Q9. SNS subscription filter policies—benefit and risk?

Benefit: fewer unwanted deliveries. Risk: misconfigured filters silently drop messages; always test negative and positive cases.

Q10. How do EventBridge targets fail closed in operations terms?

Monitor failed invocations, configure DLQs where supported on targets, alert, and ensure retries/backoff are understood per target type—do not assume fire-and-forget is observed.

Q11. What IAM mistake breaks SNS→SQS?

Missing queue resource policy allowing SNS to SendMessage from the specific topic, or KMS key policy denying the service.

Q12. How does this lesson prepare Project 3?

Project 3 combines API Gateway, Lambda, EventBridge, SQS, Step Functions, and DynamoDB into an order platform. Here you learn the messaging contracts, fan-out, DLQ, and idempotency that keep that project production-shaped.

Hands-On Exercises

Exercise: Order events with fan-out, retries, and DLQ

Problem statement. In a sandbox, build custom bus → rule → SNS → two SQS queues, with a Lambda fulfilment consumer, idempotency on orderId, and a fulfilment DLQ. Prove success, poison→DLQ, and duplicate publish safety. Budget first; full cleanup.

Tasks.

1. Create tagged bus, rule, topic, queues, DLQ, and Lambda mapping.
2. Put OrderPlaced events; show both queues receive work.
3. Force poison messages into DLQ; capture evidence.
4. Replay or republish a duplicate orderId; show idempotent behaviour.
5. Write five lines on why FIFO was or was not used.
6. Destroy all billable lab resources; evidence tagged describe calls are empty.

Evaluation rubric.

Challenge ADR

Extend the design for a constrained monthly budget and a partner HTTPS webhook that must not block learner email. Decide: SNS→SQS isolation for the partner, EventBridge vs SNS ownership for OrderPlaced, standard vs FIFO for progress, and alarm thresholds. Write a short ADR.

Decision sketch: OrderPlaced enters custom bus tlp-orders; rule targets SNS enrollment-events for fan-out; each consumer owns an SQS queue + DLQ; partner isolated; fulfilment idempotent on orderId; standard queues default; FIFO only for per-learner progress groups if out-of-order updates break the board; alarms on DLQ depth and age of oldest message; lab and non-prod retention reviewed against cost.

Consequences: more moving parts than one Lambda chain; independent failure domains; need schema ownership; FIFO limited to aggregates that truly need order.

Lab evidence checklist

Record bus name, rule name/ARN, topic ARN, queue URLs, DLQ URL, maxReceiveCount, visibility timeout, Lambda timeout, idempotency store, alarm names, PutEvents sample orderId, and cleanup timestamps.

Why this maps to Monday reliability

Enrollment week is a producer spike. Queues absorb. Idempotency absorbs retries. DLQs absorb poison. Fans-out absorb multiple teams’ delivery preferences. Without these, the API becomes the blast radius for every downstream dependency—the opposite of an application platform.

Operator notes for message replay

When replaying from DLQ or archive, prefer controlled redrive or republish of fixed payloads. Disable broken consumers first if the poison is code. Never “fix” by pointing production producers at a personal sandbox queue. Tag replay batches for audit.

Connecting to Lambda and API Gateway lessons

The order Lambda from the API lesson should publish and return quickly. Heavy work belongs to consumers. Reserved concurrency on fulfilment protects payment dependencies. Partial batch failure handling keeps one bad message from endlessly reprocessing an entire batch—implement using current Lambda SQS documentation.

Connecting to secrets

If notify workers call an email provider, the API key lives in Secrets Manager; rotation does not require changing SNS messages. Queue encryption keys need key policies that allow the worker role and, for SNS delivery, the appropriate service access—validate in docs when enabling SSE-KMS.

Deep walkthrough narrative: from checkout click to two queues

Imagine a learner completes checkout on TechLearningPro. API Gateway invokes order-accept, which validates the request, writes an ORDERS row in DynamoDB with status ACCEPTED, and calls PutEvents on bus tlp-orders with detail-type OrderPlaced. The HTTP response returns quickly with orderId. Separately, a rule matches and publishes to SNS topic enrollment-events. SNS delivers to tlp-order-fulfilment and tlp-order-notify.

The fulfilment Lambda receives a batch of one or more SQS records. Each record may wrap an SNS notification envelope—parse carefully; lab bugs often treat the SNS wrapper JSON as the domain event. The function extracts orderId, conditionally inserts into PROCESSED_ORDERS, and only then reserves a seat. If the conditional insert says the order was already processed, the function exits success so SQS deletes the message. If seat reservation fails with a transient error, the function throws; the message becomes visible again after the visibility timeout. After maxReceiveCount, SQS redrives to the DLQ and pages the order squad.

Meanwhile the notify queue feeds a different function that sends email. Email provider latency does not slow seat reservation. If notify fails permanently on a malformed address, that poison lands in notify’s DLQ without blocking fulfilment. This isolation is the architecture lesson—not the specific email SDK.

Progress updates as a second channel

Lesson completion events are noisier than orders. A learner finishing five lessons produces five progress events. Publishing each to EventBridge with source=tlp.progress and routing to a board-updater queue keeps the interactive “mark complete” API light. If the board updater is briefly down, depth grows; age-of-oldest-message alarms fire before retention. Do not write progress-only side effects inside the order fulfilment consumer—different domains deserve different rules and queues even when they share an account.

When progress must be ordered per learner-course pair, introduce a FIFO queue with MessageGroupId=learnerId#courseId. Leave order fan-out on standard queues. Mixing both patterns in one account is normal; mixing both requirements in one queue is how teams create accidental bottlenecks.

Event envelope discipline

Producers should not paste arbitrary HTTP bodies into detail. Normalize: identifiers as strings, money as integer minor units, timestamps in ISO-8601 UTC, and enumerations with documented values. Reject unknown critical fields at the consumer with validation errors that go to DLQ rather than inventing defaults (“currency missing? assume USD”) that corrupt finance reports.

Correlation IDs: include correlationId or reuse orderId in every hop so CloudWatch Logs Insights and later X-Ray lessons can stitch API → bus → queue → function. Propagation is a habit; messaging makes it mandatory because stack traces no longer connect the hops.

Redrive and replay without making it worse

SQS supports redrive from DLQ back to the source queue when you are ready. Redriving while the bug still exists refills the DLQ faster. Procedure: fix code or data, deploy, process a single message first (manual send of a cleaned payload), then redrive in controlled batches. For EventBridge archive replay, narrow time windows and expect idempotent consumers. Replay is a loaded gun—practice in sandbox during this lesson’s challenge.

Comparing “status column polling” to events

A common legacy design sets status=PENDING and runs a cron that scans for pending rows. That pattern couples scheduling to database load, scans grow with table size, and dual-writers race. Events push work when state changes; the database remains source of truth for state, not a hidden queue. If you still need a sweeper for missed events, make it a safety net with metrics, not the primary path. TechLearningPro should not discover unpaid enrollments only when a nightly job happens to run.

Security review questions for messaging

Who can events:PutEvents on the custom bus? Who can sns:Publish? Who can sqs:SendMessage versus ReceiveMessage? Are queue policies locked to specific topic ARNs? Is SSE enabled, and does the KMS policy allow SNS/EventBridge/SQS service use as required? Do DLQ dumps land in logs that are too broadly readable? Are subscription HTTPS endpoints authenticated? Answer these in the lab write-up even when the sandbox is single-account.

Capacity and quota awareness

Accounts have quotas on events per second, queue throughput, and SNS publish rates. Enrollment launches can approach quotas even when “serverless” feels infinite. Read Service Quotas for EventBridge, SNS, and SQS in your Region before a marketing send. Request increases with lead time; design producers with backoff. Do not publish inventing a “guaranteed” TPS number in an ADR—cite the quota you observed.

Cost lab discipline beyond delete

Delete event source mappings first so Lambda stops invoking. Delete rules before buses. Unsubscribe before deleting topics. Delete queues only when empty or when you accept message loss in a lab. CloudWatch Log Groups from Lambda can outlive the function and store PII from message logs—apply retention or delete per account policy. Budgets catch the forgotten mapping that republishes forever; still check the tagged resource inventory by hand once.

Anti-pattern: recursive event loops

A consumer that updates an order and republishes OrderPlaced without a guard can recurse until throttle or bill shock. Use distinct detail-types (OrderPlaced vs OrderFulfilled), or check status before publish, or use a process marker. Event-driven platforms fail dramatically when loops are accidental. Include a loop threat in design review.

How this feeds Step Functions

When fulfilment is “reserve → charge → enroll → notify” with compensations on failure, a single Lambda consumer becomes a miniature orchestrator with brittle retry logic. Keep EventBridge as the trigger: rule target = state machine. Queues still buffer notifications and analytics. Lesson 23 teaches the state machine; this lesson ensures you do not start that workflow by coupling it inside the accept API.

TechLearningPro narrative anchors (keep these vocabulary items)

Use consistently in diagrams and ADRs: order, enrollment, seat reservation, progress update, notification, Monday / enrollment week, idempotency key = orderId, DLQ owner = order squad. Shared vocabulary makes cross-team rulesets readable.

Sample design review script (15 minutes)

1. Show event JSON and pattern. 2. Show who publishes and how IAM is scoped. 3. Show fan-out topology and failure isolation. 4. Show idempotency store behavior under duplicate. 5. Show DLQ alarm and runbook link. 6. Show standard vs FIFO decision in one sentence. 7. Show cleanup evidence for non-prod. If any step is hand-wavy, the design is not ready for Project 3.

Message size, batching, and partial failure

SQS and SNS impose maximum message sizes documented by AWS—stay under them; put large artifacts in S3 and send pointers. Lambda batching from SQS improves throughput but increases blast radius when one record is poison. Enable partial batch failure responses where your runtime and event source mapping support them so successful records are deleted while failures retry. Without partial failure, one bad record can reprocess the whole batch repeatedly—watch CloudWatch for spike patterns that look like “everything fails every time” when only one learner ID is malformed.

Enrollment week load-test sketch

In non-prod, publish N synthetic OrderPlaced events with unique orderIds. Measure: accept API latency, bus invocation success, queue depth peak, fulfilment concurrency, DLQ count (should be near zero), and duplicate-key conflicts (should be zero). Then publish 10% duplicate orderIds deliberately; fulfilment success should remain correct without double seats. Finally inject 1% poison; confirm they isolate to DLQ without stalling standard queues. Capture numbers for Project 3 baselines—not as fake SLAs, as lab evidence.

When not to use messaging

Interactive reads of the progress board should not wait on a queue. Synchronous validation (“is this coupon valid?”) belongs in the request path. Ultra-low-latency feature flags should not round-trip SNS. Messaging shines for work that can complete after the user has an acceptable acknowledgment. If the product cannot acknowledge without the side effect, you need a different UX (async job status) or a true synchronous dependency with a timeout budget—document which.

Team ownership model

Assign: bus/schema owner, topic owners per domain, queue+DLQ owners per consumer squad. An unowned DLQ is an incident deferred. In ADRs, write the owner alias next to each queue. Shared “platform” topics need an intake process for new subscriptions so filter policies and fan-out cost stay intentional.

Encryption and compliance note

If a customer requires CMEK, enable SSE-KMS on queues and topics and extend key policies for SNS delivery and consumers. Message bodies may still contain learner emails—classify data and restrict who can read DLQ/replay. CloudTrail records control-plane changes to buses and rules; combine with application logs for forensics. Do not claim compliance certifications in design docs; claim controls you actually implemented.

FIFO deduplication windows

FIFO deduplication prevents duplicate publishes within the service’s deduplication interval when you supply deduplication IDs (or content-based dedup where enabled). After the window, the same ID may publish again. Consumers still need idempotency for long-term business safety. Teaching point: FIFO helps producer jitter; it does not replace a processed-orders table.

Bridging to containers

ECS/EKS workers can long-poll SQS just as Lambda does via event source mapping. The same visibility and DLQ rules apply. Prefer the same event contracts whether the consumer is Lambda or a container—Project 2 vs Project 3 should not invent parallel schemas for “OrderPlaced.”

Lesson Summary

EventBridge routes typed TechLearningPro domain events; SNS fans them out; SQS buffers work with visibility timeouts, long polling, and DLQs. At-least-once delivery demands idempotent consumers. FIFO is a scalpel for per-aggregate order, not a platform-wide default. Fan-out patterns isolate email, analytics, and seat reservation so enrollment week fails in pieces—not as one synchronous outage. Alarm on depth, age, and DLQ; tag and delete lab resources.

Step Functions and Workflow Orchestration next turns these messages into explicit sagas with retries, catches, and compensations. Keep this lesson’s buses, topics, and queues as the triggers and buffers those workflows will use—especially when you reach Project 3: Serverless Event-Driven Order Platform.

Further Reading

• Amazon EventBridge User Guide
• EventBridge event patterns
• Amazon SNS Developer Guide
• Fanout to SQS queues
• Amazon SQS Developer Guide
• SQS visibility timeout
• SQS dead-letter queues
• FIFO queues
• Using Lambda with SQS
• EventBridge quotas