The best tools to manage Kafka topics at scale split into two kinds: declarative tools that keep topics in Git, such as the Strimzi Topic Operator, Terraform, topicctl and JulieOps, and consoles that let teams create and change topics under access control, such as Conduktor, AKHQ, Kafbat UI and Kpow. Kpow is Factor House’s product, and I work at Factor House as a Solutions Architect, so it is scored on the same rubric and the same sources as every other option. At scale most teams need one of each, because a pipeline gives you the baseline and review, and a console covers incidents and the teams who do not work in the Kafka repository.
Managing topics at scale means handling creation, configuration, ownership, access and deletion across hundreds or thousands of topics and several clusters without the platform team becoming the bottleneck. This page is about choosing the tools. How to size and design the topics themselves, partition counts, replication and retention, is covered in Kafka topic and partition best practices, and the Kafka topics guide is the operational reference.
Topic lifecycle and GitOps automation
At a few dozen topics, kafka-topics.sh and a runbook work. At a few hundred, the problems are consistency and history: two topics for the same purpose with different retention, a partition count nobody can explain, and no record of who changed a config. Declarative tools answer that by making the repository the source of truth.
The pattern behind the demand is familiar. An internal thread about one prospect put it plainly: letting a user create and manage topics “required CLI/API hacks”, and they wanted “a safer UI with guardrails”. That is the gap every tool on this page is trying to close, from one side or the other.
How we score topic management tools
These six criteria decide whether a tool holds up past a few hundred topics.
1. Topics as code. Can the desired state of every topic live in a repository, be reviewed as a pull request and be applied by CI? This is how consistency survives team turnover.
2. Guardrails on creation. The costly topic mistakes are defaults that look harmless on day one. In our operational issues talk I called out two. Retention set too high on a high-throughput topic “is a slow-burn problem”, because disk fills over weeks until it is suddenly urgent, and “Running out of disk is about the worst state Kafka can be in”. And a generous max.message.bytes “is a trap that tends to spring later”, when the topic becomes popular. A good tool enforces naming, partition, retention and size limits at creation.
3. Scoped self-service with approval. Teams should create and change their own topics without being able to touch another team’s. Our comparison of Kafka RBAC tools covers how each tool scopes access. Tom Crowley, our founding engineer, describes Kpow’s answer to the approval part in one line: “We have optional approval via staged mutations”. The alternative, a ticket to the platform team for every topic, is the bottleneck the search is usually about.
4. Audit. Who created, changed or deleted a topic, and when. Without it, a missing topic or a changed retention is a forensics exercise.
5. Multi-cluster reach. Dev, staging and several production clusters, ideally from one place. On cluster count itself my view is that fewer, larger shared clusters beat one per team: in the talk’s Q&A I said splitting clusters along governance domains is inefficient because those domains keep changing, and better governance and audit controls are the better answer. That makes the topic tooling on a shared cluster matter more, not less.
6. API and bulk operations. At scale, clicking through a UI one topic at a time does not work. The tool needs an API that fits into an internal developer portal or pipeline, and a safe way to act on many topics at once.
The tools compared
Every row uses the same fields in the same order. Cells say why, and the Source column points at each tool’s own documentation.
| Tool | Topics as code | Creation guardrails | Scoped self-service and approval | Audit | Multi-cluster | API and bulk | Source |
|---|---|---|---|---|---|---|---|
| kafka-topics.sh and kafka-configs.sh | Only if you script it. | None beyond cluster defaults. | None. Kafka ACLs decide who can create topics. | None in the tool. | One cluster per invocation. | Scriptable, one topic per call. | Apache Kafka operations |
| Strimzi Topic Operator | Strong. One KafkaTopic resource per topic, and changes made outside the resource are reverted. |
Kubernetes admission controls or CI checks you add. | Kubernetes RBAC and namespaces. | Git and Kubernetes history. | One operator per Kafka cluster. | Kubernetes API. Topics without a KafkaTopic resource are left alone. |
Strimzi Topic Operator |
| Terraform (Mongey provider) | Strong. kafka_topic, with import for existing topics. |
Terraform validation and policy tooling you add. | Git review. | Git and plan history. | One provider block per cluster. | Terraform workflow. Also manages ACLs, quotas and SCRAM users. | terraform-provider-kafka |
| topicctl | Strong. YAML per topic, applied with apply, plus replica placement strategies. |
Partial. Declared settings per topic, and it refuses to rebalance topics whose partition count or retention differs from the YAML. | Git review, with a confirmation required for every mutation. | Git history. | A cluster config per cluster, checked against the cluster ID before apply. | CLI and a read-only REPL. Topics are never deleted by apply. | topicctl |
| JulieOps | Strong for topics, ACLs, schemas and connectors. | Partial. Reusable config plans give topics named service levels. | Git review. | Git history. | One topology per cluster. | CLI in CI. The maintainer describes the project as in hibernation, last push June 2024. | JulieOps |
| Conduktor Self-service | Strong. Every change goes into Git. | Strong. Policies such as maximum partitions and naming conventions. | Strong. Applications own resources, and access requests go to the owning team. | Git history. | Part of the Conduktor platform. | API and GitOps resources. | Conduktor’s documentation, “Kafka self-service: topic requests and governed access” |
| AKHQ | None. | None beyond RBAC scope. | Partial. RBAC with regex patterns on resource names. | Partial. Opt-in audit events to a Kafka topic for topic create, config change, partition increase and delete. | Yes, several clusters in one instance. | REST API. | AKHQ audit docs |
| Kafbat UI | None. | None beyond RBAC scope. | Partial. RBAC with regex values on topic names. | Partial. Opt-in audit log to a topic or the console. | Yes, several clusters in one instance. | REST API. | Kafbat UI RBAC docs |
| Kpow | None. Kpow is not a config-as-code tool, though its create form shows the equivalent kafka-topics.sh command. |
Partial. Tenants limit creation to topics matching a name, prefix or suffix. There are no retention or config limits in the current release. | Strong. RBAC with allow, deny or stage per role and resource, tenants that scope what each role sees, and staged mutations for admin approval. | Strong. Every mutation, including bulk actions, is written to the audit log, with webhook delivery. | Strong. Up to 12 clusters per instance. | Strong. REST API, and bulk actions across many topics gated by their own permission. | Kpow topic docs |
Scoring notes. On creation guardrails, Conduktor’s policies are ahead of Kpow, which today limits creation by topic name through tenants and does not enforce retention or config limits. On topics as code, the Strimzi Topic Operator and Terraform are ahead of every console, Kpow included. Kpow is strongest where many teams share clusters: scoped visibility, staged approval, audit and bulk operations across up to 12 clusters.
Kpow live demo
Try governed topic management
Use the Kpow demo to browse topics, their config sources and the topic create form that generates the matching kafka-topics.sh command.
For platform teams sharing clusters across many teams.
Try the Kpow demoTopics as code
Strimzi Topic Operator
What it is. A Kubernetes operator that manages one Kafka topic per KafkaTopic resource.
Where it wins. The Strimzi documentation states that when a KafkaTopic exists, “any configuration changes made outside the resource are reverted”, which gives you real drift correction, not only drift detection. It also leaves topics without a resource alone, so you can adopt it gradually.
Where it falls short. Strimzi-managed Kafka on Kubernetes only, and Kafka topic names that are not valid Kubernetes names need the separate spec.topicName field.
Terraform with the Mongey provider
What it is. The open-source Terraform provider for Kafka, with kafka_topic, kafka_acl, kafka_quota and kafka_user_scram_credential resources.
Where it wins. Topics sit next to the rest of your infrastructure code, existing topics can be imported, and a plan shows every change before it applies.
Where it falls short. Guardrails are whatever policy tooling you add around Terraform, and a topic created outside Terraform stays invisible to it until someone imports it.
topicctl
What it is. Segment’s open-source tool for declarative topic management, inspired by kubectl.
Where it wins. Its safety properties are written for exactly this job: user confirmation for every mutation, topics never deleted by apply, partitions added but never removed, interruptible and idempotent runs, per-cluster locking and a cluster ID check before apply.
Where it falls short. It manages topics and ACLs from YAML, but ownership, approval and self-service live in your Git process.
JulieOps
What it is. A GitOps tool, formerly Kafka Topology Builder, for topics, access control, schemas and connectors, with reusable configuration plans.
Where it wins. Plans let you define named service levels, such as a retention and message size profile, and apply them to many topics, which is a practical guardrail.
Where it falls short. Its maintainer’s note says the project is “mostly on a long winter hibernation”, so check its activity before building on it.
User interfaces and self-service portals
Conduktor Self-service
What it is. Part of Conduktor’s commercial platform. Its documentation describes applications that own their topics and ACLs, policies that constrain requests such as maximum partitions and naming conventions, and access requests routed to the owning team, with every change stored in Git.
Where it wins. The most complete creation guardrails and ownership model in this list.
Where it falls short. It is a commercial platform decision, not a standalone topic tool, and the ownership model has to be set up before teams see the benefit.
AKHQ
What it is. A free, self-hosted Kafka UI. Its RBAC groups can restrict roles to resources matching regular expressions, and its audit configuration can emit events for topic creation, configuration changes, partition increases and deletion to a Kafka topic.
Where it wins. Free, widely used, and scoped access by name pattern.
Where it falls short. No creation guardrails or approval step, and auditing is off until configured.
Kafbat UI
What it is. The actively maintained fork of the former Provectus Kafka UI, with topic creation and configuration, RBAC with regex values on topic names, and an opt-in audit log.
Where it wins. Free, self-hosted and quick to deploy.
Where it falls short. No creation guardrails or approval step. Access is scoped but not staged.
Kpow
What it is. Factor House’s Kafka management and monitoring product.
Where it wins. Governed operations across teams and clusters. Tenants restrict each role to topics by name, prefix or suffix, and a user can only create topics valid for their tenant. RBAC policies can stage an action instead of allowing it, so a topic request waits for an administrator. Bulk actions run one mutation across many resources, such as deleting 300 topics in a request, under their own permission. How to keep deletions like that safe is covered in our comparison of tools for controlling destructive Kafka operations. Every mutation lands in the audit log. The topic create form suggests the five most common values used on your cluster for each config, can copy another topic’s config, and shows the equivalent kafka-topics.sh command if you would rather run it through your pipeline.
Where it falls short. It does not enforce retention or config limits on creation in the current release, and it is not a config-as-code tool. Pair it with a declarative baseline if you need one.
Rebalancing as topics grow
Topics at scale bring partition placement problems with them: new brokers that receive none of the existing partitions, and brokers that end up leading more than their share. The Kafka CLI moves partitions, and Cruise Control plans and executes rebalances against goals such as rack awareness and disk balance. Our comparison of partition reassignment tools scores the options on planning, throttling and cancellation. Kpow reassigns individual partitions and runs leader elections from a topic’s page. Kafka cluster management covers where reassignment fits in day-to-day operations.
Schema registry and evolution
Topic management at scale includes the schemas on those topics. Confluent Schema Registry and Apicurio Registry enforce Avro, Protobuf or JSON Schema compatibility so a producer cannot publish data that breaks consumers. JulieOps can declare schemas alongside topics, and Kpow, AKHQ and Kafbat UI all connect to a schema registry to browse and manage subjects. The Kpow schema documentation covers what it supports, and our comparison of schema registry management tools covers the wider choice.
How Factor House approaches topic management at scale
We built Kpow for clusters shared by many teams, where the question is less “how do I create a topic” and more “who is allowed to create which topics, who approved it, and what changed”. Tenants give each team a view of only its own topics, staged mutations put an approval step in front of creation or deletion, bulk actions handle cleanup across hundreds of topics, and the audit log records all of it. The Kpow multi-tenancy documentation shows how tenants are defined in the RBAC configuration. To test the topic side in practice, open the Kpow demo: filter the topic list, open a topic’s configuration with each value’s source, and walk through the create form with its common values and generated kafka-topics.sh command.
What we have not built is retention and config guardrails on creation. For a hard limit on retention or partition count today, enforce it in a pipeline with Strimzi, Terraform or topicctl, and use Kpow for visibility, scoped access and audit across the clusters. For the design decisions the guardrails should encode, Kafka topic vs partition explains why partition count is the one you cannot easily undo.
How to choose
One platform team, topics in Git. Strimzi’s Topic Operator on Kubernetes, otherwise Terraform or topicctl.
Many teams creating topics, with hard policy limits. A self-service platform with policies, such as Conduktor, or a pipeline with policy checks in CI.
Many teams on shared clusters, with a need for scoped access, approval and audit. A console with tenancy, staged approval and an audit log, such as Kpow, alongside whatever declarative baseline you keep.
Small team, low budget. AKHQ or Kafbat UI with auditing turned on. Our broader Kafka management tools comparison covers these consoles beyond topic management.
FAQ
What is the best way to manage Kafka topics as code?
On Kubernetes with Strimzi, the Topic Operator’s KafkaTopic resource, which also reverts changes made outside it. Elsewhere, the Mongey Terraform provider or topicctl. All three keep the desired state in Git and apply it through a reviewed change.
How do I stop teams creating badly configured Kafka topics?
Enforce limits where topics are created. In a pipeline, that means CI checks on the YAML or Terraform. In a console, it means policies (Conduktor), name-scoped permissions (AKHQ, Kafbat UI, Kpow tenants) and an approval step (Kpow staged mutations). Set sensible cluster defaults as the last line of defence.
Can I manage Kafka topics across multiple clusters from one tool?
Yes. Kpow manages up to 12 clusters per instance, and AKHQ and Kafbat UI both support several clusters. Declarative tools such as Terraform and topicctl handle multiple clusters through one configuration per cluster.
Is there a Kafka topic management tool with an audit log?
Kpow records every mutation in its audit log. AKHQ and Kafbat UI can write audit events to a Kafka topic once auditing is enabled. Declarative tools rely on Git history, which covers only changes that went through the pipeline.