Testing Spring Kafka

Introduction

What is Testing Spring Kafka? Spring Kafka tests use EmbeddedKafka for lightweight tests or Testcontainers for production-like broker behavior.

Understanding the topic

What happens — Testing Spring Kafka:

• Add spring-kafka dependency.
• Configure bootstrap servers in application.yml.
• Use KafkaTemplate to send.
• Use @KafkaListener to consume.

Visual explanation

Pipeline view:

KafkaTemplate
  ↓
topic
  ↓
@KafkaListener container
  ↓
service logic
  ↓
ack/retry/DLT

Step-by-step explanation

1. Configure — Producer/consumer factories.
2. Produce — kafkaTemplate.send().
3. Consume — @KafkaListener method.
4. Handle errors — Retry topics + DLT.

Informative example

Example:

kafka-topics --bootstrap-server localhost:9092 --create --topic testing-spring-kafka --partitions 6 --replication-factor 3
kafka-console-producer --bootstrap-server localhost:9092 --topic testing-spring-kafka
kafka-console-consumer --bootstrap-server localhost:9092 --topic testing-spring-kafka --from-beginning
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group testing-spring-kafka-service

Execution workflow

Best practices

• Use typed event classes and schema validation.
• Use manual or record ack modes for critical processing.
• Add retry topics and DLT with alerts.
• Test with embedded Kafka or Testcontainers.

Common mistakes

• Doing slow blocking work inside listener threads without concurrency planning.
• Catching exceptions and still acknowledging bad records.
• No DLT monitoring.

Summary

Testing Spring Kafka — Configure listener containers intentionally: ack mode, concurrency, error handling, retries, DLT, observability, and idempotent processing.