Kafka in Docker is the standard way to get a local development cluster, and the fastest way to ship a misconfiguration if you treat the container as a deployment. This page covers both halves: getting the local stack right, and knowing where the container stops being enough. It assumes you know what Kafka is; the production tutorial covers the operating layer.
Core needs
Running Kafka in Docker means three decisions: which image, which Compose topology, and how the listeners are configured so anything outside the Docker network can actually connect.
The image comes from an official registry: the Apache Kafka image or Confluent’s images are the maintained options, and modern tags run KRaft mode so no ZooKeeper container is needed. The habit of grabbing whichever image a tutorial happens to use has a real cost, because containerised Kafka tooling goes stale: running an abandoned image means running software with known, unpatched security vulnerabilities.
Docker Compose is the standard way to run a complete local environment. A realistic template runs either a single broker for quick tests or three brokers to mirror production replication, plus the supporting services: a schema registry, Kafka Connect and a UI. A well-built stack is one docker compose pull and docker compose up away from a working environment. The idea is old and proven: Spotify open-sourced a combined Kafka and ZooKeeper Docker image years ago precisely to give developers a local Kafka for testing.
Networking is where local Kafka fails first. A client connects to the bootstrap server, then follows the addresses the broker advertises for each partition leader. Inside Docker those advertised addresses default to container hostnames that resolve nowhere on the host, so KAFKA_ADVERTISED_LISTENERS must expose one listener for traffic inside the Docker network and a second on localhost for clients outside it. Nearly every “connection refused from my app but the container is running” report is this.
The reason your Docker Kafka deserves real care: it is one of three clusters, not a toy beside the real one. Almost every team running Kafka in production runs at least three clusters, development, staging and production, and the Docker Compose stack IS the development cluster for most of them. A Compose file that diverges from production, different image lineage, different security posture, no schema registry, quietly moves your integration testing onto a system that does not resemble the one you ship to.
Which leads to the practice I hold teams to even in Compose: treat the configuration as code. Version the Compose file, the broker configs and the listener setup like any other source, and change them through review. Broker configuration changes should be version-controlled and applied through CI in production, and the habit starts, or dies, in the development stack.
Production warnings
A single-node Docker Kafka is a development tool, not a deployment target. Kafka is stateful software: without volume mounts its entire log sits in the container’s writable layer and dies with the container, and without memory limits and resource tuning it competes with everything else on the host. Container defaults are sized for laptops, and real deployments size deliberately: published container guidance for Kafka tooling starts at figures like 1 CPU and 2Gi of memory per service, and some UIs default to a 4 GB RAM minimum in their Docker configuration.
Disk is the risk that containers make easier to hit. Running out of disk is about the worst state a Kafka broker can be in and is very hard to recover from, and a containerised broker on an undersized volume gets there quickly. Retention settings and volume sizing have to be decided together.
Orchestrating containerised Kafka at production scale means Kubernetes, and Kafka’s stateful nature is exactly what makes that hard: persistent volume sizing, storage class selection and pod anti-affinity all need specific knowledge that a stateless service never demands. The KRaft migration removed ZooKeeper from that picture, which is one less stateful system to run, but the brokers themselves still carry all of the above.
The honest framing for all of it: the failure pattern the production tutorial describes, misconfiguration rather than software defect, applies double in containers. Docker does not change the failure modes. It just makes the misconfiguration faster to ship.
The cost of getting containerised state wrong is documented in dollars: one post-incident analysis of a production Kubernetes-hosted Kafka failure put it at $240,000 in SLA penalties over 62 minutes. And storage choice is exactly the kind of decision that causes it. Grab replaced an earlier Kafka storage design built on NVMe instance store volumes because those volumes could not survive worker node replacement without manual intervention: fast disks, wrong lifecycle. In containers, the storage question is never only “is it fast”, it is “what happens to it when the node goes away”.
When something does go wrong, my operational rule is one change at a time. Increase disk and watch it settle, then raise network throughput and watch that. A cluster that just hurt you is exactly the wrong place to apply three fixes at once, because when the graphs move you will not know which change moved them. Containers make bundling changes temptingly easy, one new image, five config diffs, and the discipline matters more there, not less.