When we reach for it
Event-driven integration between several systems, and pipelines where being able to replay history — after a bug, or for a new consumer — is worth the infrastructure.
Kafka engineering
A durable, replayable event log that lets systems communicate without knowing about each other, and lets a consumer that was down catch up rather than lose the messages.
When we would argue against it
Two services that need to talk do not need Kafka. It is a substantial operational commitment, and we have seen more of them installed for the architecture diagram than for the throughput.
What it looks like in delivery
Schemas versioned and compatibility enforced, consumers built to be idempotent, and dead-letter handling designed at the start. Most event-driven systems fail on redelivery, which is a certainty rather than an edge case.
Where this appears on the site
Published work that names it
Derived from the stacks published on those pages, not written here — so this list cannot claim something the page it points at does not say.
Also in data and storage
Working in Kafka?
Tell us what it is running, what it costs you today, and what you need it to do next. A senior engineer will tell you what we would keep and what we would change.