ZooKeeper vs KRaft

Introduction

What is ZooKeeper vs KRaft? Older Kafka clusters used ZooKeeper for metadata coordination.

Understanding the topic

What happens — ZooKeeper vs KRaft:

• Old clusters used ZooKeeper for metadata.
• KRaft embeds metadata in Kafka itself.
• Controller quorum manages leaders.
• New deployments should use KRaft.

Visual explanation

Pipeline view:

Producer
  ↓
Topic partition log
  ↓
Consumer group A
  ↓
Consumer group B
Replay is possible because records are retained

Step-by-step explanation

1. Elect controller — Raft consensus.
2. Store metadata — Topics, partitions, ACLs.
3. Notify brokers — Leader changes.
4. Migrate — ZK → KRaft upgrade path.

Informative example

Example:

kafka-topics --bootstrap-server localhost:9092 --create --topic zookeeper-vs-kraft --partitions 6 --replication-factor 3
kafka-console-producer --bootstrap-server localhost:9092 --topic zookeeper-vs-kraft
kafka-console-consumer --bootstrap-server localhost:9092 --topic zookeeper-vs-kraft --from-beginning
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group zookeeper-vs-kraft-service

Execution workflow

Best practices

• Design events around business facts: OrderCreated, PaymentAuthorized, InventoryReserved.
• Pick keys based on ordering and load distribution.
• Document owner, retention, schema, partitions, and consumers for every topic.
• Assume duplicate delivery and build idempotent consumers early.

Common mistakes

• Expecting global ordering across a topic.
• Creating one partition because local demos work.
• Skipping schema governance until multiple teams deploy independently.

Summary

ZooKeeper vs KRaft — Start every design with event ownership, topic naming, key choice, schema contract, retention, replay plan, and monitoring.