Best tools to manage Kafka broker configs

The tools for managing Kafka broker configs fall into four groups: Kafka’s own kafka-configs.sh and Admin API, configuration-as-code tools such as Strimzi and Terraform, Kafka UIs such as AKHQ and Kafbat UI, and management consoles such as Confluent Control Center and Kpow. Kpow is Factor House’s product, and I work at Factor House as a Solutions Architect, so it is scored on the same rubric and the same sources as every other option. The CLI is the baseline every cluster has. The others add what it lacks: a record of intent in Git, a readable view of what is actually running, and control over who can change it.

A broker config is the set of properties a broker reads from server.properties at startup, plus the subset Kafka lets you change while the broker runs. For how brokers fit into the cluster as a whole, start with the complete Kafka guide or Kafka brokers in production.

How Kafka broker configuration works

Every broker property has an update mode, listed in the Apache broker configuration reference. The mode decides whether a tool can change it without a restart, whatever tool you use:

read-only     requires a broker restart
per-broker    may be updated dynamically for each broker
cluster-wide  may be updated dynamically as a cluster-wide default

When the same property is set at more than one level, Kafka applies this order of precedence: a dynamic per-broker value, then a dynamic cluster-wide default, then the static value in server.properties, then Kafka’s own default. Dynamic values are stored in the cluster metadata, not in server.properties, so reading the file on disk does not tell you what a broker is running.

kafka-configs.sh is the reference tool for all of it. These are the commands from the Apache guide to updating broker configs:

Describe the dynamic overrides on broker 0:

kafka-configs.sh --bootstrap-server <broker:9092> --entity-type brokers --entity-name 0 --describe

Change a per-broker value, then remove the override so the broker falls back to the static or default value:

kafka-configs.sh --bootstrap-server <broker:9092> --entity-type brokers --entity-name 0 --alter --add-config log.cleaner.threads=2
kafka-configs.sh --bootstrap-server <broker:9092> --entity-type brokers --entity-name 0 --alter --delete-config log.cleaner.threads

Set and describe a cluster-wide default that every broker picks up:

kafka-configs.sh --bootstrap-server <broker:9092> --entity-type brokers --entity-default --alter --add-config log.cleaner.threads=2
kafka-configs.sh --bootstrap-server <broker:9092> --entity-type brokers --entity-default --describe

--describe on its own lists dynamic configs only. Adding --all lists every config for the entity, “including static configs if available”, per the option definition in the Kafka source. Use --all when you want to know what a broker is actually running.

Thread pool sizes are a special case worth knowing before you change them. num.io.threads, num.network.threads and num.replica.fetchers can be changed dynamically as a cluster default, but Apache restricts each update to between half and double the current size.

How we score broker config tools

These criteria come from an incident I walked through in our operational issues talk. A team raised replica fetcher and I/O thread counts on a managed cluster, the upgrade failed and rolled back, and the thread counts did not roll back with everything else. Six months later a disk failed, the rebuild saturated disk I/O, and producer timeouts spread across teams. From the talk: “It took someone looking specifically at that broker’s config to notice it didn’t match the vendor’s best practices”. Each criterion below is something that would have caught it sooner.

1. Seeing the running config and its source. You need every property a broker is running, and whether each one is a default, a static value or a dynamic override. In the talk I recommended making a dump of the current broker config part of your runbooks, so you check it matches what you expect rather than assume it does. A tool that only shows server.properties, or only dynamic overrides, answers half the question.

2. Catching drift against a baseline. My advice from the same session: “After any rollback, diff the running config against your baseline, and periodically audit the running config against your documented best practices”. A tool scores well here if it keeps a declared baseline and either reconciles to it or shows you where the cluster differs.

3. Changing config as code. “Treat broker config changes like code where you can: version-controlled and applied through CI.” That gives review before the change and history after it.

4. The break-glass path. Even teams with GitOps keep a way to change config by hand in an incident. From the talk: “some environments keep a break-glass option for emergency config changes that never quite makes it back into GitOps.” A tool scores well if it makes the out-of-band change visible and attributable, so it can be reconciled later.

5. Access control and audit. Who can change a broker property, and whether there is a record of who did it and when. Our comparisons of Kafka RBAC tools and Kafka audit logging tools go deeper on each half. In Kpow’s case, as Tom Crowley, our founding engineer, has described it, the audit trail is persisted in Kpow’s internal audit log, which is itself backed by a Kafka topic.

6. Safe change mechanics. Whether the tool respects update modes, makes scope obvious (one broker or the whole cluster), and encourages the rule from the same talk: “one change at a time, monitored before the next”.

F1 The lesson from a config change that stayed half-applied for six months

A rollback isn't done until every change is verified and reverted, and config drift is silent until it isn't.

Chad Harris, Solutions Architect at Factor House

The tools compared

Every row uses the same fields in the same order. The Source column points at the tool’s own documentation.

Tool	Running config and source	Drift against a baseline	Config as code	Break-glass visibility	Access control and audit	Scope and fit	Source
kafka-configs.sh and Admin API	Strong. --describe --all shows every config for a broker. Output is text per broker, one call each.	Manual. You script the diff against your baseline yourself.	Strong if you build it. Scripts in a repo applied by CI are the common pattern.	None. A change from a laptop leaves no record unless your shell or CI keeps one.	None in the tool. Kafka ACLs control who can alter configs, and there is no change log.	Every Kafka cluster, per-broker and cluster-wide.	Apache Kafka docs
Strimzi (spec.kafka.config)	Partial. The declared config is in the Kafka resource. The running values are still read with the CLI.	Strong. The operator validates and applies the config during reconciliation.	Strong. Broker config is a Kubernetes resource in Git.	Partial. A manual change made outside the resource is not in Git, so it has to be caught by comparing against the running config.	Kubernetes RBAC on the resource, with Git history as the audit trail. Some properties, including listeners, log directories and authorization, are managed by Strimzi and cannot be set.	Strimzi-managed Kafka on Kubernetes only.	Strimzi documentation
Terraform	Depends on the provider.	Strong where supported, through plan and apply.	Strong where supported.	Shows up as drift on the next plan.	Git and your Terraform workflow.	The open-source Mongey provider manages topics, ACLs, quotas and SCRAM users, not broker configs. Confluent’s provider has a cluster config resource for Confluent Cloud Dedicated clusters only.	terraform-provider-kafka
JulieOps	Not covered.	Not covered for brokers.	Strong for topics, ACLs, schemas and connectors, not broker configs.	Not covered.	Git and CI review.	Its maintainer describes the project as in hibernation, with the last push in June 2024.	JulieOps
AKHQ	Partial. Broker config is viewable under the NODE resource.	None.	None.	Partial. Changes through the UI are role-gated.	Partial. RBAC has READ_CONFIG and ALTER_CONFIG on nodes. Its audit events cover topic config changes, and the documented list has no broker config event.	Free, self-hosted, Apache 2.0.	AKHQ docs
Kafbat UI	Partial. Broker config is viewable and editable under the clusterconfig permission.	None.	None.	Partial. UI changes can be audited when auditing is enabled.	Partial. clusterconfig has view and edit actions, and an opt-in audit log writes operations to a topic or the console.	Free, self-hosted, Apache 2.0.	Kafbat UI RBAC docs
Confluent Control Center	Strong for Confluent Platform. Shows cluster defaults, with an indicator where a setting was modified from the default.	None built in.	None. Changes apply directly to the cluster.	Partial. Dynamic editing can be disabled organization-wide.	Tied to Confluent Platform RBAC.	Confluent Platform only.	Confluent’s documentation, “Manage Kafka Clusters Using Control Center for Confluent Platform”
Kpow	Strong. Every broker’s configuration in one table, filterable by broker, property, source, importance and whether it is read-only, across up to 12 clusters per instance.	Partial. Source filtering shows every non-default value at a glance, but Kpow does not store a declared baseline or diff against one.	None. Kpow is not a config-as-code tool.	Strong. A change made through Kpow is recorded in the audit log, and the source column shows dynamic overrides however they were made.	Strong. Editing needs the BROKER_EDIT permission under RBAC, and every mutation is written to the audit log, which can also go to a webhook.	Any Kafka cluster Kpow connects to, including managed services. Licensed per cluster, with a free Community Edition for up to three clusters.	Kpow broker docs

Scoring notes. For config as code on vanilla Apache Kafka, the honest winner is a repository of kafka-configs.sh scripts or server.properties templates applied by CI, because the popular GitOps tools stop at topics and ACLs. Strimzi is the strongest option if you already run Kafka on Kubernetes. Kpow is strongest on seeing what is running and who changed it, and it does not replace a declared baseline.

Kpow live demo

See every broker config and its source

Open the Kpow demo's Broker Configuration table and filter by source to find every non-default value on a live cluster.

For operators who need to know what a broker is actually running.

Try the Kpow demo→

GitOps and configuration as code

What they are. Tools that keep the desired state in Git and apply it through CI, so every change has a review and a history.

Where they win. Review before the change, history after it, and a baseline you can diff against. For broker config, that is exactly what the treat-it-like-code advice above asks for.

Where they fall short. The open-source tools most often recommended for Kafka GitOps manage topics and ACLs, not brokers. The Mongey Terraform provider lists kafka_topic, kafka_acl, kafka_quota and kafka_user_scram_credential as its resources. JulieOps covers topics, access control, schemas, connectors and related metadata, and its maintainer’s note says the project is “mostly on a long winter hibernation”. kafka-gitops describes itself as managing “Apache Kafka topics and ACLs”. Confluent’s Terraform provider does have a confluent_kafka_cluster_config resource, but its own documentation limits it to Dedicated clusters on Confluent Cloud.

What to use instead. For broker config on vanilla Kafka, version server.properties for static values and a script of kafka-configs.sh --alter calls for dynamic ones, apply both from CI, and run --describe --all against every broker on a schedule to catch what changed outside it.

Kubernetes-native management

What it is. Strimzi defines broker configuration as key-value pairs under spec.kafka.config in the Kafka custom resource, then “validates and applies these properties during reconciliation.”

Where it wins. Broker config becomes a Kubernetes resource like everything else in the cluster, with GitOps history and reconciliation built in. It is the only option here that gives vanilla Apache Kafka a genuine declared baseline for broker settings.

Where it falls short. It only applies to Strimzi-managed clusters, and Strimzi reserves some properties for itself, including node.id, log.dirs, listeners, authentication and authorization, so you cannot set those through config. Its own guidance is to adjust configuration incrementally and monitor the impact using broker and client metrics, which is the same discipline the CLI needs.

Enterprise control planes and UIs

These are the tools people reach for during an incident, when the question is “what is this broker running right now, and can I change it without logging into a box”.

Native CLI alternatives

What it is. kafka-configs.sh and the Admin API it wraps, in every Kafka distribution.

Where it wins. It is the only tool guaranteed to be present, it reads the broker directly, and --describe --all is the most complete answer to “what is this broker running”. Scripted, it is also the base of most config-as-code setups for vanilla Kafka.

Where it falls short. One entity per call, text output, no record of who ran it, and nothing stops two people changing the same property from two laptops. The Apache precedence rules also mean a static value you edited in server.properties can be silently overridden by an old dynamic value still stored in the cluster, which is worth checking with --describe before a restart. JVM settings such as heap size sit outside broker config entirely, in the start-up environment, and container memory limits can change how the JVM sizes itself, as our notes on Amazon Corretto 11 memory issues show for a JVM running in Docker.

How Factor House approaches broker config

We built the broker configuration view in Kpow for the moment an incident needs an answer to “what is this broker actually running”. It shows every property with its source and importance, lets someone with the right role change a dynamic value without shell access, and records the change in the audit log. The Kpow broker management documentation covers the filters and edit workflow. The quickest way to test the criteria on this page is the Kpow demo: open the Broker Configuration table on a live cluster, filter by source to list every non-default value, and compare the same property across brokers.

What we recommend alongside it is the same discipline from the talk: keep the baseline in Git, apply changes through CI where you can, make one change at a time, and check the running config after every rollback. If you are also running Flink next to Kafka, Factor Platform brings both under one control plane.

How to choose

One cluster, one team, vanilla Kafka. Version your config in Git, apply it with kafka-configs.sh from CI, and schedule a --describe --all diff.

Kafka on Kubernetes. Strimzi’s spec.kafka.config gives you a declared baseline with reconciliation.

Confluent Platform or Confluent Cloud Dedicated. Control Center or Confluent’s Terraform provider respectively.

Several clusters, several teams, and a need to prove who changed what. Add a console with role-based editing and an audit log that covers broker changes. Check that claim in each tool’s own documentation, since AKHQ’s audit list does not include broker config. Kafka broker monitoring covers the metrics that tell you a config change has gone wrong. For the wider tool landscape, see our broader Kafka management tools comparison.

FAQ

What is the difference between static and dynamic Kafka broker configs?

A static (read-only) config is read from server.properties at startup and needs a broker restart to change. A dynamic config can be changed while the broker runs with kafka-configs.sh --alter or the Admin API, either per broker or as a cluster-wide default. The update mode of each property is listed in the Apache broker configuration reference.

How do I see the current config of a Kafka broker?

Run kafka-configs.sh --bootstrap-server <broker:9092> --entity-type brokers --entity-name <id> --describe --all. Without --all you only see dynamic overrides. A management console such as Kpow shows the same information for every broker in one table, with the source of each value.

Can I manage Kafka broker configs with Terraform?

Only on some platforms. The open-source Mongey provider manages topics, ACLs, quotas and SCRAM users, not broker configs. Confluent’s provider can update cluster configs on Confluent Cloud Dedicated clusters. On Kubernetes, Strimzi’s Kafka resource is the closest equivalent for vanilla Kafka.

How do I audit who changed a Kafka broker config?

Kafka itself does not record it. You need a tool in the change path that does: CI history if changes only go through a pipeline, or a console with an audit log that covers broker changes, such as Kpow or Kafbat UI with auditing enabled.

Related reading

• Kafka: the complete guide
• Kafka brokers in production
• Kafka broker monitoring
• Kafka stream governance
• Kafka KRaft
• Best tools for Kafka audit logging