KRaft is Kafka’s built-in consensus protocol, and it removes ZooKeeper from the architecture entirely. Cluster metadata moves into an internal Kafka topic, __cluster_metadata, managed by a quorum of controller nodes running the Raft protocol. One system to deploy, secure and monitor instead of two.
The timeline matters because it sets your upgrade path. KRaft became production-ready in Kafka 3.3, reached full feature parity with ZooKeeper mode in Kafka 3.9, and is mandatory from Kafka 4.0. A 3.x cluster still on ZooKeeper must migrate to KRaft before it can upgrade to 4.x. This page covers the migration, the limits, the configuration and the day-2 operations in that order. The wider context sits in the Kafka architecture guide and the hub.
Migration guides and cluster sizing
Migration is no longer optional. Kafka 3.x clusters running with ZooKeeper must complete the KRaft migration before upgrading to 4.x, so the question is when and how, not whether. The migration replaces ZooKeeper’s role with the controller quorum and moves all cluster metadata into the event-sourced __cluster_metadata log.
For a greenfield cluster the sizing answer is settled. The recommended baseline is Kafka 4.0 or later, KRaft only, in isolated mode: three dedicated controller nodes plus three or more brokers, spread across three availability zones. The controller quorum stays at three nodes for most clusters, five where higher availability is required.
Monitoring changes with the migration. ZooKeeper-era control plane metrics are replaced by KRaft-native equivalents, including consensus metrics under kafka.server:type=raft-metrics and a new family of metadata loader and controller queue metrics. A migration plan that does not update dashboards and alerts ships a cluster nobody can see.
Two facts calibrate how scary the migration should feel. The live ZooKeeper-to-KRaft migration path has been GA since Kafka 3.6, September 2023, so this is a supported, walked path rather than a leap. And the largest known migration is done: Confluent Cloud moved thousands of clusters to KRaft without breaching SLAs. The protocol is not the risk.
The tooling around your cluster is. With Kafka 4.x in KRaft mode, one widely used open-source UI was still showing inconsistent partition leaders on every page refresh, because it queried brokers instead of the KRaft quorum controller. That is the migration lesson in one bug: audit everything that reads cluster metadata, dashboards, UIs, automation scripts, before you flip, because tools written in the ZooKeeper era make ZooKeeper-era assumptions.
Production stability and limits
Feature parity is done. KRaft reached full parity with ZooKeeper mode in Kafka 3.9, which closes the checklist question that stalled early adopters.
The scale story is the reason KRaft exists. The ZooKeeper architecture imposed a hard ceiling: metadata handling degraded substantially as partition counts grew past roughly 200,000 per cluster, with the old rules of thumb at 4,000 partitions per broker. KRaft’s consolidated metadata management removes that bottleneck. Lab tests have demonstrated stable operation at 2 million partitions per cluster, and real-world production clusters typically run in the hundreds of thousands.
Hold the two numbers apart when someone waves the 2-million-partition benchmark at a capacity meeting: that is a lab demonstrating what the metadata layer can survive, not what your brokers can serve. The per-broker economics of partitions did not change, which is why a practical bound of 10,000 to 20,000 partitions per broker remains sensible for most workloads even on KRaft: partition count still costs memory, file handles and recovery time on every broker regardless of where the metadata lives.
The ecosystem lag shows up here too. One long-standing cluster manager was immediately incompatible with a KRaft test cluster, reported by a practitioner who could not remove it from their stack. Feature parity in the broker does not mean feature parity in your tooling, and the tools are where migrations actually stall.
Configuration and architecture
The first configuration decision is node roles. Every KRaft node declares process.roles as broker, controller, or both. Combined mode is fine for development stacks, and production runs isolated roles: an odd number of dedicated controller nodes, separate from the brokers, so the consensus layer is insulated from broker-side I/O pressure and garbage collection pauses.
The controller quorum has hard requirements. A production deployment needs a minimum of three controller nodes for quorum, five for higher availability, on dedicated nodes with fast SSD storage, spread across separate availability zones, with the controller listener isolated from client listeners. The quorum is defined in controller.quorum.voters, listing each controller’s node.id and address, and every node’s identity is fixed by its node.id.
Under the quorum sits an event-sourced metadata log, and it is what makes the scale claims real: Raft-based metadata management supports millions of partitions. Its health is directly observable, with MetadataErrorCount (kafka.controller:type=KafkaController,name=MetadataErrorCount) counting failed metadata log operations, which should be zero.
On broker sizing around the quorum, my advice runs against the instinct to buy the biggest boxes: avoid the largest available nodes. Mid to large brokers leave you headroom to scale vertically when you need capacity fast, and once you are on the largest instance the only move left is horizontal scaling, which on Kafka means partition movement at the worst possible moment. Keep a gear in reserve.
And a configuration discipline that KRaft makes easier to follow: after any rollback, diff the running configuration against your baseline, and audit it against documented best practice periodically rather than after incidents. Config drift is the quiet failure mode of long-lived clusters, a rollback that half-applied, an emergency override nobody removed, and the whole class of problem is invisible unless something is comparing running state to intended state. The metadata log gives you a single place where cluster configuration lives. Use it.
Operations and day-2 management
Day-2 KRaft operations start with watching the quorum. Quorum health metrics from kafka.server:type=raft-metrics belong in monitoring from day one, alongside the KRaft-specific MBeans that appear with the mode: FencedBrokerCount and LastAppliedRecordLagMs, the second of which tells you how far a node has fallen behind the metadata log. The ZooKeeper-related MBeans they replace are simply gone, so old dashboards silently show nothing.
The cluster-level alert set does not change. Under-replicated partitions, ActiveControllerCount and OfflinePartitionsCount remain the most impactful signals, with ActiveControllerCount now describing the KRaft controller: since Kafka 4.0 it is the KRaft controller that manages cluster metadata and leader election, and exactly one controller must be active.
Security gets simpler, and that is a genuine operational win. KRaft provides a single security model, removing the separate SASL and ACL configuration that ZooKeeper required, which eliminates a second attack surface. The remaining work is securing the inter-controller channel with TLS and SASL like any other listener, instead of maintaining a parallel ZooKeeper security regime.
The day-2 point I push hardest: a managed KRaft cluster is not an outsourced KRaft cluster. Managed Kafka health is a shared responsibility, and it needs your own monitoring and configuration hygiene, because your provider watches their SLA, not your consumer lag, and certainly not whether your quorum’s metadata log is applying cleanly. The one metric I would never delegate: a metadata-apply-error-count above zero means metadata corruption on the broker, and no managed-service status page is going to tell you that.
For a sense of what disciplined day-2 operations look like at the far end, JPMorgan Chase runs more than 100 Kafka clusters and dealt with the operational burden by building a health index, a centralised control plane, and an orchestrated patching pipeline: monitoring, patching and quota enforcement, automated. Most estates need a fraction of that machinery. Every estate needs some deliberate version of it, and KRaft simplifying the architecture is precisely the moment to build it, while the pieces are one system instead of two.