Acknowledgements

Introduction

What is Acknowledgements? acks controls when a producer considers a write successful: 0 means fire-and-forget, 1 means leader ack, all means leader plus in-sync replicas.

Understanding the topic

What happens — Acknowledgements:

• acks=0: fire-and-forget, fastest, may lose data.
• acks=1: leader confirms, limited durability.
• acks=all: all ISR replicas confirm — safest.
• Pair acks=all with min.insync.replicas on broker.

Visual explanation

Pipeline view:

App creates message (e.g. "User signed up")
  ↓
Producer sends to topic (e.g. user_signups)
  ↓
Kafka stores in partitioned log (replicated)
  ↓
Consumers / analytics / alerts read in real time

Step-by-step explanation

1. Initialization — Producer connects to bootstrap servers and receives metadata (topics, partitions, leaders).
2. Message creation — Application builds a ProducerRecord with topic, key, value, and optional timestamp.
3. Serialization — Key and value serializers convert objects to bytes (e.g. StringSerializer).
4. Partitioning — Hash of key picks partition; no key → round-robin / sticky partitioner.
5. Batching — Records accumulate in the RecordAccumulator per partition for efficient network use.
6. Sending — Background sender thread ships batches to the partition leader broker.
7. Acknowledgement — Broker responds; producer retries on failure based on acks and retry settings.

Informative example

Example:

kafka-topics --bootstrap-server localhost:9092 --create --topic acknowledgements --partitions 6 --replication-factor 3
kafka-console-producer --bootstrap-server localhost:9092 --topic acknowledgements
kafka-console-consumer --bootstrap-server localhost:9092 --topic acknowledgements --from-beginning
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group acknowledgements-service

Execution workflow

Best practices

• Use acks=all with min.insync.replicas for critical events.
• Enable idempotence for retry safety.
• Choose keys intentionally; random keys destroy ordering.
• Monitor record-error-rate, request-latency, batch-size, and retries.

Common mistakes

• Ignoring asynchronous send failures.
• Using null keys for events that require aggregate ordering.
• Making producer timeouts too low and failing during normal broker leader movement.

Summary

Acknowledgements — Use acks=all, enable idempotence, choose meaningful keys, and handle send callbacks for errors.