Most Kafka incidents we get paged for come down to five concepts: partitions, leaders, the in-sync replica set, producer acks and consumer group assignment. This is the mental model we use to read the symptoms and find the cause fast.
Kafka has no message priority, so a marketing blast on the same topic can delay password reset codes by twenty minutes. We compare the designs that actually work: separate topics with dedicated consumers, pausing bulk partitions in one consumer, and broker quotas.
Our enrichment consumers rebalanced every few minutes, lag climbed past two million messages and some records were processed twice. A slow external API pushed batch processing past max.poll.interval.ms. How we proved it, fixed the poll loop and made rebalances cheap.