Kafka

Introduction

Apache Kafka is a distributed event streaming platform — the backbone of event-driven microservices. Producers publish to topics; consumers in consumer groups process messages with at-least-once or exactly-once semantics.

Informative example

Spring Kafka producer and consumer:

// Producer
@Service
public class OrderEventPublisher {
    private final KafkaTemplate<String, OrderEvent> kafka;
    public void publish(OrderEvent event) {
        kafka.send("order-events", event.orderId(), event);
    }
}

// Consumer
@KafkaListener(topics = "order-events", groupId = "inventory-service")
public void handle(OrderEvent event) {
    inventoryService.reserveStock(event.sku(), event.qty());
}

// Config
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.auto-offset-reset=earliest

Best practices

• Design topics around domain events, not CRUD tables.
• Make consumers idempotent — at-least-once means duplicates happen.
• Monitor consumer lag — it's the #1 Kafka ops metric.

Purpose of this lesson

Master Kafka so you can apply it confidently in production Java code, technical interviews, and code reviews.

Step-by-step explanation

1. Understand the core idea behind Kafka.
2. Walk through the runnable example and tweak it in the playground.
3. Apply the pattern in a small Spring Boot or CLI exercise of your own.
4. Re-read the common mistakes and interview Q&A to lock the concept in.

Interactive workflow diagram

Debugging tips

• Consumer lag growing? Check processing time, increase partitions/consumers, or investigate slow handler.
• Duplicate messages? Expected with at-least-once — make handlers idempotent.

Optimization strategies

• Profile before optimizing — JFR (Java Flight Recorder) and async-profiler reveal real hotspots.
• Prefer immutable data and stream pipelines over hand-rolled loops when readability matters.
• Reach for the right JDK collection (ArrayList vs LinkedList vs ArrayDeque) before writing custom data structures.

Enterprise example

Teams at Netflix, Uber and Goldman Sachs apply Kafka daily — usually wrapped behind Spring Boot services with observability hooks (Micrometer + OpenTelemetry).

Interview questions & answers

Summary

In this lesson you learned Kafka — the concept, syntax, a runnable example, and the production pitfalls to avoid. Apply it in the playground before moving on.