#kraft

Data Engineering

Kafka internals for debugging: partitions, replicas, ISR, acks and consumer groups

John Nguyen · Jun 15, 2026 · 8 min read

Most Kafka incidents we get paged for come down to five concepts: partitions, leaders, the in-sync replica set, producer acks and consumer group assignment. This is the mental model we use to read the symptoms and find the cause fast.

Data Engineering

Why Kafka projects fail: eight failure modes we have seen and how to avoid them

John Nguyen · Oct 19, 2024 · 7 min read

Kafka rarely fails because the broker is buggy. Projects fail because teams pick it for the wrong job, skip schemas, guess at partitions, assume exactly-once, and leave operations and security for later. Here are the eight patterns we keep seeing and what to do instead.