Kafka Interview Questions and Answers (2026)
35 Kafka interview questions with answers, from basics to scenario questions for experienced engineers: partitions, ISR, acks, consumer groups, KRaft.
On this page · 7 sections
These are the Kafka questions that come up in backend, data engineering and system design interviews, with answers you can say out loud in a minute or two. They’re grouped from fundamentals to the scenario questions interviewers ask experienced engineers. Answers reflect current Kafka (4.x, KRaft only); where older versions behaved differently, it’s called out, because interviewers often probe exactly that.
For the full picture behind the answers, see Kafka architecture explained and the free Kafka course.
Kafka basics
1. What is Apache Kafka?
A distributed, replicated, append-only log. Producers write records to topics; brokers store them durably on disk for a configured retention period; any number of consumer groups read them independently at their own pace. It’s used for event streaming, decoupling services, log and metrics pipelines, change data capture and stream processing.
2. How is Kafka different from a traditional message queue like RabbitMQ?
In a queue, a message is typically removed once a consumer acknowledges it, and the broker tracks delivery per message. Kafka keeps records for the retention period whether or not they’ve been read; each consumer group tracks its own position (offset). That makes replay, multiple independent readers and very high throughput easy, at the cost of per-message routing features. See Kafka vs RabbitMQ for details.
3. What are topics, partitions and offsets?
A topic is a named stream. It’s split into partitions; each partition is an ordered, append-only log stored on brokers. Each record in a partition has a sequential offset. Partitions are the unit of parallelism and of ordering.
4. Does Kafka guarantee ordering?
Only within a partition. To keep events for one entity in order, give them the same key so they land in the same partition. There is no ordering across partitions.
5. How does a producer choose a partition?
If the record has a key, the default partitioner hashes the key (murmur2) modulo the number of partitions. Without a key, the producer uses sticky partitioning: it fills a batch for one partition, then moves on. You can also set the partition explicitly or plug in a custom partitioner. Watch out in mixed-language systems: librdkafka-based clients hash keys with CRC32 by default, so the same key can land elsewhere (details and a test).
6. What is a broker, and what is a cluster?
A broker is one Kafka server that stores partitions and serves clients. A cluster is a set of brokers sharing metadata. Clients connect to any broker (bootstrap.servers), fetch metadata, then talk directly to the leader of each partition they need.
7. What is a consumer group?
A set of consumers sharing a group.id. Kafka assigns each partition to exactly one consumer in the group, so the group processes a topic in parallel while each partition stays ordered. Different groups read the same topic independently.
8. What happens if a group has more consumers than partitions?
The extra consumers sit idle. Parallelism within a group is capped by the partition count, which is why partition count is a capacity decision.
Replication and durability
9. What are leader and follower replicas?
Each partition has replication.factor copies. One is the leader and handles all writes (and by default reads); followers fetch from the leader to stay in sync. If the leader fails, an in-sync follower is elected leader.
10. What is the ISR?
The in-sync replica set: replicas that are caught up with the leader. A follower that hasn’t caught up within replica.lag.time.max.ms (30 seconds by default) is removed from the ISR. Only ISR members can become leader without losing acknowledged data (unless unclean election is enabled).
11. Explain acks=0, acks=1 and acks=all.
acks=0: the producer doesn’t wait; records can be lost silently. acks=1: the leader has written the record; it’s lost if the leader dies before followers copy it. acks=all: every ISR member has it. Since Kafka 3.0 the producer default is acks=all with idempotence enabled.
12. What does min.insync.replicas do?
It sets the minimum ISR size for an acks=all write to succeed. With replication factor 3 and min.insync.replicas=2, writes continue with one broker down; with two down, producers get NotEnoughReplicas errors instead of a false promise of durability. acks=all without it can degrade to one copy.
13. What is the high watermark?
The offset up to which all ISR replicas have the data. Records below it are committed, and consumers can only read committed records, which is why an acknowledged acks=all write survives a leader failover.
14. What is unclean leader election?
Electing a replica that is not in the ISR when no ISR member is available. It restores availability but loses records the old leader had acknowledged. unclean.leader.election.enable is false by default; turn it on only where availability beats correctness, such as some metrics topics.
Producers
15. How does the producer achieve high throughput?
Batching per partition (batch.size, linger.ms), compression of whole batches (lz4, zstd, snappy, gzip), multiple in-flight requests, and asynchronous sends with callbacks. A few milliseconds of linger.ms usually multiplies throughput.
16. What is an idempotent producer?
The broker assigns the producer an ID, and the producer numbers each batch per partition. If a retry re-sends a batch that was actually written, the broker detects the duplicate sequence number and drops it. That gives exactly-once writes per partition per producer session, and it’s on by default in modern clients.
17. What are Kafka transactions used for?
Writing to several partitions (and committing consumer offsets) atomically. A consume-transform-produce app reads with isolation.level=read_committed, writes outputs and offsets in one transaction, and either everything becomes visible or nothing does. This is how Kafka Streams provides exactly-once processing.
Consumers
18. How do consumer offsets work?
The group commits the offset of the next record to read for each partition. Committed offsets are stored in the internal __consumer_offsets topic, managed by the group coordinator broker. After a restart or rebalance, consumers resume from the committed offset.
19. Auto-commit vs manual commit?
Auto-commit (enable.auto.commit=true) commits periodically in the background, so a crash can lose records (committed but not processed) or replay them. Manual commit after processing gives at-least-once: on a crash you reprocess some records, so processing should be idempotent.
20. At-most-once, at-least-once, exactly-once: how do you get each?
At-most-once: commit before processing. At-least-once: commit after processing (most common). Exactly-once: idempotent producer plus transactions with read_committed consumers inside Kafka, or at-least-once plus idempotent writes (upserts, dedupe keys) when the output is an external system.
21. What is a rebalance, and why is it a problem?
Reassigning partitions when consumers join, leave, crash or miss heartbeats. With the classic eager protocol, every consumer stops while partitions are reassigned. Cooperative incremental rebalancing (the CooperativeStickyAssignor) only moves the partitions that need to move. Kafka 4.0 made the new consumer group protocol (KIP-848) generally available: with group.protocol=consumer, the broker computes assignments incrementally and the group never pauses as a whole.
22. What do max.poll.interval.ms and session.timeout.ms control?
session.timeout.ms: how long without heartbeats before the consumer is considered dead. max.poll.interval.ms (5 minutes by default): the maximum gap between poll() calls; exceed it (slow processing) and the consumer is removed from the group, causing a rebalance. Slow handlers should use smaller max.poll.records or hand work to other threads.
23. What is consumer lag?
Log-end offset minus committed offset, per partition. Growing lag means consumers can’t keep up. It’s the most important Kafka health metric; the lag management lesson covers monitoring and fixes.
Storage and operations
24. How does Kafka store data on disk?
Each partition is a directory of segment files plus offset and time indexes. Writes append to the active segment; old segments are deleted by retention (retention.ms, retention.bytes) or compacted. Kafka relies on sequential I/O and the OS page cache, and can send data to consumers with zero-copy transfers (for non-TLS connections).
25. What is log compaction?
A cleanup policy (cleanup.policy=compact) that keeps at least the latest record for each key and removes older ones; a record with a null value (a tombstone) deletes the key. Used for changelogs and state: the topic becomes a durable table of latest values.
26. What is KRaft, and why did Kafka drop ZooKeeper?
KRaft is Kafka’s built-in metadata quorum: a few controller nodes run the Raft protocol over the internal __cluster_metadata log. Kafka 4.0 removed ZooKeeper entirely. Benefits: one system to operate and secure, much faster controller failover, and support for far more partitions. Older clusters must migrate to KRaft on a 3.x release before upgrading to 4.x.
27. What does the controller do?
Tracks broker liveness, elects partition leaders, handles topic and partition changes, and publishes metadata to brokers. In KRaft, one controller in the quorum is active and the rest are hot standbys.
28. Can you reduce the number of partitions of a topic?
No; you can only add partitions. Adding them changes the key-to-partition mapping, so ordering by key breaks for keys that move. To reduce partitions you create a new topic and migrate.
29. How do you choose the number of partitions?
At least the number of consumers you want working in parallel at peak, with headroom for growth; also consider target throughput per partition. Don’t over-partition: each partition adds files, memory, and leader elections during failures.
30. What is tiered storage?
Moving older log segments to object storage (such as S3) while recent data stays on broker disks (KIP-405, production-ready since Kafka 3.9). It decouples retention from local disk size and speeds up broker replacement.
Scenario questions for experienced engineers
31. A consumer group’s lag keeps growing. How do you debug it?
Check whether lag grows on all partitions (consumers too slow overall: add consumers up to the partition count, batch work, speed up downstream calls) or a few (a hot key or a stuck consumer). Look for frequent rebalances (processing exceeding max.poll.interval.ms), GC pauses, and downstream latency. Fixes include more partitions, async processing with careful offset commits, and fixing hot keys.
32. You need strict ordering per customer but also high throughput. What do you do?
Key by customer ID so each customer’s events share a partition, and use enough partitions for parallelism. Inside a consumer, process different keys in parallel but each key sequentially (for example, a worker per key hash), committing offsets only up to the lowest fully processed record.
33. How do you avoid duplicates when a consumer writes to a database?
Assume at-least-once and make the write idempotent: upsert by a natural key, or store processed event IDs (or the partition-offset pair) in the same database transaction as the result, and skip anything already seen.
34. A broker dies. What happens step by step?
The controller notices missed heartbeats, removes the broker from ISRs, and elects new leaders from the remaining in-sync replicas for partitions it led. Producers and consumers refresh metadata and move to the new leaders. With replication factor 3 and min.insync.replicas=2, acks=all writes keep working. When the broker returns it catches up and rejoins the ISRs, and preferred leaders are restored automatically. The failure walkthrough shows real output from a three-node cluster.
35. How would you design an event pipeline that must never lose orders?
Producers with acks=all, idempotence on and retries; topics with replication factor 3, min.insync.replicas=2, unclean election off, replicas spread across racks or zones (broker.rack); consumers committing after processing with idempotent writes; lag and under-replicated-partition alerts; and an outbox pattern on the producing service so the database write and the event can’t diverge.
How to prepare
- Read Kafka architecture explained until you can draw the diagram from memory.
- Run a local cluster and break it: the hands-on exercises in the Kafka course (consumer groups, rebalancing, production).
- Practise explaining where Kafka fits in a design with the message queues lesson of the System Design course.