Retry Topics

Introduction

What is Retry Topics? Retry topics move failed records through delayed retry topics before sending them to a dead letter topic.

Understanding the topic

What happens — Retry Topics:

• Classify retryable vs poison errors.
• Retry with backoff for transient failures.
• Route poison messages to DLT.
• Alert and replay from DLT after fix.

Visual explanation

Pipeline view:

KafkaTemplate
  ↓
topic
  ↓
@KafkaListener container
  ↓
service logic
  ↓
ack/retry/DLT

Step-by-step explanation

1. Catch error — In listener or poll loop.
2. Retry — Backoff N times.
3. DLT — Publish with failure metadata.
4. Alert — Page on DLT growth.

Informative example

Example:

kafka-topics --bootstrap-server localhost:9092 --create --topic retry-topics --partitions 6 --replication-factor 3
kafka-console-producer --bootstrap-server localhost:9092 --topic retry-topics
kafka-console-consumer --bootstrap-server localhost:9092 --topic retry-topics --from-beginning
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group retry-topics-service

Execution workflow

Best practices

• Use typed event classes and schema validation.
• Use manual or record ack modes for critical processing.
• Add retry topics and DLT with alerts.
• Test with embedded Kafka or Testcontainers.

Common mistakes

• Doing slow blocking work inside listener threads without concurrency planning.
• Catching exceptions and still acknowledging bad records.
• No DLT monitoring.

Summary

Retry Topics — Configure listener containers intentionally: ack mode, concurrency, error handling, retries, DLT, observability, and idempotent processing.