Best Kafka management tools for payments companies

Tools compared

Rank	Tool	Total (out of 100)	Out of the data path	Production access on request	Audit trail per person	Who can see unmasked data	Directory and Kafka sign-in	Many teams, shared clusters	Cost a year (modelled)
1	Kpow	87	One container, state in your Kafka, not a proxy	Time-boxed temporary policies via API, staged approvals	Every action with the IdP user, data queries included	Server-side, show last four, no per-role exemption	SAML, OIDC, LDAP; Kerberos, SCRAM or mTLS to brokers	Tenants and per-action RBAC	$20,880
2	Kafbat UI	64	One stateless container	Per-resource RBAC, no approvals	Optional, reads at level ALL, no view	Server-side, the same for every viewer	OAuth2, OIDC, LDAP; no SAML	Per-resource roles per cluster	$11,520
3	AKHQ	56	One stateless container	Regex groups, UI-only if JWT secret unset	Opt-in, no reads, no view	Global filters, the same for every viewer	LDAP, OIDC; no SAML	Groups by resource and cluster pattern	$16,320
4	Lenses	53	HQ on PostgreSQL, agent and database per cluster	Strict global masking, no approvals	In-product audit log from Team tier	Strictest, no exemption even for admins	SSO incl. Entra ID and Okta	Roles on groups only	$2,880 plus quoted licence
5	Confluent Control Center	35	Dedicated host, broker reporter JAR	Confluent RBAC, no DENY, no approvals	Broker principal, not always the person	No masking described	OIDC on self-managed	Admin access only, per TD	$2,880 plus quoted subscription
6	Kafdrop	29	One stateless process	None, anyone who can reach it	None	No masking	None built in; proxy in front	One cluster, no roles	$11,520
7	Conduktor	58	Console on PostgreSQL; Gateway proxy in the data path	Cross-team access requests, no expiring grant	70+ event types with user, in the UI	Console exemptions per user or group	LDAP, OIDC	Per user or group, most permissive grant wins	$122,880; $212,880 with Gateway Core and Protect

No tool meets every column, and payments companies commonly pair a management tool for people with broker ACLs for services and with encryption or tokenisation of card numbers before they reach Kafka at all.

The tools, ranked for payments companies

What payments companies need from a Kafka management tool

This page is about payments companies specifically: card networks, acquirers and processors, and payment terminal and gateway providers, whose Kafka topics carry card and transaction data. For the regulation-by-regulation view across banks, payments and insurance, with DORA, PCI DSS, GDPR and SOX mapped to Kafka controls, see best Kafka governance tools for financial services. For a bank’s platform team serving hundreds of projects on shared clusters, see best Kafka management tools for banks.

Payments companies run some of the largest Kafka fleets described in public. PayPal, which is not named here as a Kpow user, described its fleet in 2023 as more than 85 clusters and over 1,500 brokers hosting over 20,000 topics, with traffic peaking at about 1.3 trillion messages a day on Retail Friday 2022. Its clusters are deployed across security zones according to data classification, and access is only through the SASL port, with each application declaring whether it produces or consumes. How PayPal uses Apache Kafka in production covers the rest of that architecture.

Payments differ from other Kafka users because of the card number. PCI DSS, the standard published by the PCI Security Standards Council, applies to the cardholder data environment, which the Council’s glossary defines as the system components, people and processes that store, process or transmit cardholder data, together with components that have unrestricted connectivity to them. The same glossary defines masking as concealing a segment of the card number when it is displayed, for use when there is no business need to see all of it. A Kafka tool that shows authorization or dispute topics to engineers is displaying card data, so it has to mask it for the people who do not need the full number and has to record who looked.

The Council’s guidance on scoping (December 2016, written against PCI DSS 3.2) also brings into scope every system that can connect to the cardholder data environment, so any management tool that reads card topics is assessed with it. Where the tool sits changes how much is assessed. A proxy that producers and consumers connect through transmits every card message that passes it, so it is part of the cardholder data environment, and where authorization or settlement messages travel over Kafka, its latency, availability and configuration also become part of how payments are processed. A tool that connects beside the brokers as an ordinary Kafka client is assessed as one more client, and an outage in it stops people from looking at Kafka without stopping payments.

Around that sit the constraints most payments companies share. Engineers debug a failed payment by reading the messages for one transaction across several topics. People are managed in a central identity provider, several product teams share clusters, and a company that has grown by acquisition can end up running more than one Kafka distribution.

Each of the six criteria this page scores comes from those needs, and for each one PCI DSS or a public engineering source says something specific that general lists of Kafka tools leave out. The numbered PCI DSS requirements below are cited from Microsoft’s published guidance on the standard, which restates each requirement in full, because the standard itself sits behind the Council’s document library licence.

Out of the data path. A proxy has to be sized for the traffic it carries, because every request from every client routed through it passes it, while a tool that connects as a Kafka client reads metadata and touches records only when a person asks for them. Kai Waehner’s review of Kafka proxies says a proxy has to be deployed in a highly available and horizontally scalable way, and payments traffic makes that sizing harder than an average suggests, because PayPal’s figure of about 1.3 trillion messages a day was a Retail Friday peak, and a component on the payment path has to be built for that day. Conduktor accepts that cost for what it buys, since its encryption, masking of the data itself, policy enforcement on client traffic and Virtual Clusters run through Conduktor Gateway, a proxy that can encrypt or mask card fields for applications as well as people, while Conduktor Console connects to clusters directly, masks in its own UI and needs a PostgreSQL database. Kpow gives people RBAC, masking in inspection, temporary access, tenants and an audit log without putting anything in front of the brokers, and it does not attempt to protect data for applications, so the trade runs in both directions. Staying out of the data path is also not unique to Kpow, since Kafbat UI, AKHQ and Kafdrop score the same 9 on this criterion, and what separates those tools is what each can do from that position. In PCI DSS terms neither design takes the tool out of the assessment, because the scoping guidance brings in every system that connects to the cardholder data environment, and what differs is whether the component can slow or stop a payment.

Production access on request. PCI DSS is specific about how people may read stored card data. Requirement 7.2.6, as Microsoft’s guidance on PCI DSS Requirement 7 restates it, restricts user access to query repositories of stored cardholder data to applications or other programmatic methods, with access and allowed actions based on user roles and least privilege, and lets only the responsible administrators query those repositories directly. A topic that retains card numbers holds them for the whole of its retention period, so on a plain reading the usual production path of VPN, jump box and a console consumer typed by hand is direct query access by people who are not its administrators, with the further problems that the jump box typically holds full cluster access and nothing records which commands were run. A management tool with roles is the application route the requirement describes. Requirement 7.2.4 on the same page asks for all user accounts and access privileges to be reviewed at least once every six months, and that is where the length of a grant matters, because a grant that names one topic and expires when the incident ends leaves no standing privilege for the half-yearly review to find. Kafka itself cannot express such a grant, because a Kafka ACL allows or denies an operation for a principal on a resource and carries no end time, so on Kafka an expiring grant has to come from the management tool.

Audit trail per person. When engineers work through a shared tool, the broker sees the tool’s service account, and PCI DSS treats that arrangement as something to be controlled. Requirement 8.2.2 allows shared or generic accounts only on an exception basis, and Requirement 8.6.1 says that where an account used by a system or application can be used interactively, every action taken has to be attributable to an individual user; Microsoft’s guidance on PCI DSS Requirement 8 restates both. A Kafka tool through which people read topics under its own principal is interactive use of an application account, so the tool’s own log is the only place that attribution can come from. Requirement 10 then says what the log has to hold: audit logs capture all individual user access to cardholder data (10.2.1.1), which through a Kafka tool is mostly the data queries people run, each entry records the user, the type of event, the date and time, whether it succeeded, where it originated and the data or resource affected (10.2.2), and the history is retained for at least 12 months with the most recent three months immediately available for analysis (10.5.1), all as restated in Microsoft’s guidance on PCI DSS Requirement 10. The retention period is far longer than a Kafka topic’s default, so where a tool keeps its trail on a topic, as Kpow does, the company has to raise that topic’s retention or send the records to its SIEM.

Who can see unmasked data. The glossary’s definition of masking covers what a screen shows, and the Council’s note on 8-digit BINs and PCI DSS adds how much: masking and truncation formats should display or retain only the minimum number of digits needed for the specific business need, and its example is a customer service agent who needs only the last four digits to verify a card. An engineer finding out why a payment failed needs no more than that, which is what a last-four policy leaves visible. Without masking in the tool, the only safe policy for a card topic is to deny inspection, and engineers then wait on a ticket while someone on the platform team extracts and cleans the records by hand, a slow process that can itself expose data. The harder design question is the exception, because the glossary ties masking to there being no business need to view the whole number, and some teams, such as fraud or chargeback operations, may have that need. A tool whose masking is the same for every viewer forces a choice between masking the topic for those teams too and leaving it clear for everyone, which is why Conduktor Console’s exemptions per user or group earn the highest score on this criterion and Kpow’s per-topic policies earn 6. Masking is a display control in every one of these tools, and the same glossary entry points to truncation for a number that is stored, processed or transmitted, so no masked view changes what a topic holds.

Directory and Kafka sign-in. Requirement 8.4.2, restated in Microsoft’s guidance on PCI DSS Requirement 8, asks for multi-factor authentication on all access into the cardholder data environment, and opening a card topic in a Kafka tool is access of that kind. A Kafka tool normally gets its second factor from the identity provider it hands sign-in to, so SAML or OpenID Connect sign-in is the route by which the tool meets the requirement, and a tool with no sign-in of its own, such as Kafdrop, relies entirely on the proxy placed in front of it. The same guidance restates Requirement 8.2.5, that access for terminated users is revoked immediately, and Requirement 8.2.6, that inactive accounts are removed or disabled within 90 days. Both are met in the directory when a tool reads roles from directory groups, and both have to be repeated by hand in a tool that keeps local accounts. The broker side matters as well, because the tool’s configuration holds a credential for every cluster it manages. On Amazon MSK that credential can be an AWS identity, since IAM access control for Amazon MSK authenticates clients with IAM, and Kpow’s Amazon MSK configuration connects with AWS credentials without a separate username and password pair or certificate for the brokers.

Many teams, shared clusters. Whether card data gets clusters of its own is the company’s decision, because the Council’s scoping guidance says network segmentation of the cardholder data environment is not a PCI DSS requirement and recommends it as a way to reduce the scope and cost of an assessment. A talk on cluster consolidation describes the same reasoning from the operator’s side, which is that regulated companies move sensitive workloads to their own clusters because segregation is much easier to prove to an auditor that way, not because a regulation requires it. The talk also names the cost of sharing in the other direction, where analytical workloads placed on a low-latency cluster pay low-latency prices for capacity they do not need. However the clusters are divided, several teams end up on each one, and Apache Kafka’s multi-tenancy documentation covers how they are kept apart with topic naming, authentication, ACLs and quotas, the last of which lets a platform team cap a reporting client so that it cannot take broker capacity from the authorization path. The management tool is best scoped along the same boundaries, because an instance that reaches card clusters is in scope under the scoping guidance, with its sign-in and all of its users, so a company that has segmented its card clusters can run a separate instance inside that zone and keep engineers who only work on other clusters out of it, and within each instance a tenant per team limits what acquiring, issuing and fraud each see.

What payments companies use Kpow for

Four payments companies that run Kpow are named here: a card network, an acquirer, a terminal maker and a payments processor. How each one uses it is described only where it has been stated in public; for the others, the card carries public information about the company itself. None of them is quoted on its Kafka architecture. The rubric below the rankings is built from what PCI DSS asks of any tool that can display card data, and each card is tagged with the rubric criteria its evidence speaks to.

How a payments company runs its Kafka with Kpow

The workflows below are how a payments company’s platform team puts Kpow to work on clusters that carry card data. Each one is built from documented Kpow features.

Install it beside the clusters. Kpow runs as one Docker container or Java JAR, or on Kubernetes with the Helm charts, inside the company’s own network. It needs no external database, because snapshots, metrics and the audit log are held in topics on the company’s own clusters, and it connects to brokers with the same cluster security settings as any other client. Producers and consumers on the payment path keep connecting to the brokers directly. One instance manages up to 12 clusters across self-managed Apache Kafka, Confluent Platform, Confluent Cloud and Amazon MSK. Where the software runs also decides who else joins the assessment. The PCI SSC glossary defines a service provider as a business directly involved in processing, storing or transmitting cardholder data on another entity’s behalf, or one that provides services that control or could impact the security of that data, so a console hosted by its vendor that reads card topics makes the vendor a party the company has to manage as one, while software the company runs itself is one more component in its own assessment. Kafka data stays in the company’s environment unless the company connects one of Kpow’s optional AI model integrations to a hosted model provider. Patching belongs to the same picture, because Requirement 6.3.3, restated in Microsoft’s guidance on PCI DSS Requirement 6, asks for critical and high security patches to be installed within one month of release on every system component, and a tool that is one container with no database is patched by replacing the image.

Mask card numbers to the last four digits. Data policies redact fields in Data Inspect and ksqlDB results on the server, in keys, values and headers, including fields nested inside Avro, Protobuf and JSON records. The ShowLast4 function, used in the documentation’s own credit card example, shows a card number as its last four digits, and where a function cannot apply to a field, Kpow falls back to full redaction rather than showing the value. While policies are active, String SerDes are removed from Data Inspect so nobody can read a card topic around them.

Trace one transaction across topics. When an authorization fails, an engineer opens Data Inspect and filters the authorization, settlement and dispute topics for one transaction ID with a kJQ filter, with card fields still masked. The query itself is recorded in the audit log. Inspection is also how a platform team checks what producers are writing into card topics. The PCI SSC glossary describes sensitive authentication data, which includes card verification codes, full track data and PINs, as data that might be transmitted or processed as part of a payment transaction but not stored, and Apache Kafka keeps every record for the whole of a topic’s retention period whether or not it has been consumed. A verification code that one producer includes in an authorization event is therefore held on the brokers long after the authorization has completed, whatever a tool masks on screen, and searching the authorization topics for those fields is how the team finds that producer.

Grant production access for one incident. An engineer who needs to read a production topic raises a request in the company’s change system. Once it is approved, that system calls the Kpow API to create a temporary policy that grants inspect access on the named topic for a fixed time. Temporary policies joined the admin API endpoints in release 94.1, and self-service Kafka governance with Kpow and ServiceNow walks through the pattern. The policy expires on its own, is capped at seven days by default, and is recorded in the audit log. TD Bank’s platform team describes this pattern in its talk on running Kafka at bank scale: an engineer fills in a ServiceNow form, the form calls the Kpow API, and access arrives within a minute and lasts an hour or two, which gives the bank an audit trail of every grant. The same mechanism covers emergencies, where the break-glass account is usually a shared one. Requirement 8.2.2 permits a shared account only for an exceptional circumstance, with use limited to the time needed, a documented business justification and management approval. AWS’s guidance on temporary elevated access treats break-glass access in a time-critical emergency as one more case for a time-bound grant, and a temporary policy created from an incident ticket for a named engineer meets the same need without a shared account at all.

Give each team its own view. The platform team adds a tenant for each team, so acquiring, issuing and fraud each see their own topics and consumer groups on a shared cluster, and only the Connect clusters and schema registries the tenant includes, and maps the team’s directory group to a role through SAML, OpenID Connect or LDAP. RBAC sets Allow, Deny or Stage per action and resource, with Deny winning where policies overlap.

Hold risky changes for a second person. With staged mutations, a role can be set to Stage on actions such as resetting a settlement consumer group’s offsets or deleting a topic, so the request waits in Kpow until an administrator approves or denies it, and the full history lands in the audit log. A staged request expires after 15 minutes by default, a window the company can lengthen with the scheduler setting if approvals go through a change board. For a payments company the reason to hold these actions is that an offset reset or a message produced by hand can move money a second time. An offset reset on a settlement or payout consumer group makes the group read the same records again, and Apache Kafka guarantees at-least-once delivery by default, so a replay is safe only where the consumer is idempotent. Stripe’s engineering post on idempotency describes the usual defence, an idempotency key that lets a charge be retried without the customer being charged twice, and the guide to dead letter queues in Kafka makes the same point about replay, that a record whose processing starts a payment starts it again unless the consumer checks for it. A reset does not have to be deliberate either, because a consumer group that is renamed has no committed offsets, and with auto.offset.reset set to earliest it reads the topic from the beginning. Producing by hand is the other case, since a message typed into a payment topic is an instruction that no upstream system created. TD Bank’s platform team says in its talk that nobody should publish to a production topic, and that anyone who has to must ask for permission and an exception and explain why and when that will change. In Kpow the equivalent is to leave produce on production payment topics out of every standing role, so that it is denied by default, and to grant it through a temporary policy when an exception is approved. Failed payment messages usually go through retry topics before a dead letter topic, the design Uber’s engineering team describes with a payment as its example, and clone to topic lets RBAC restrict both ends of a replay, so that only records from a consumer’s own dead letter topic can be copied, and only into that consumer’s retry topic.

Show the assessor who did what. The audit log records each action, data inspect queries included, with the user from the company’s directory and the policy that allowed it. Kpow shows the last seven days in the product and writes every record to the __oprtr_audit_log topic on the company’s own cluster, where Kpow’s topics default to one week of retention, which the company can raise to cover its assessment period, and webhooks send mutations, data inspect queries or both to Slack, Microsoft Teams or the company’s SIEM for long-term retention. Because the record is a Kafka topic, the ACLs on that topic decide who could alter it, and a tenant that excludes Kpow’s internal topics, as the documented Kpow Hidden tenant does, keeps the audit topic out of an engineer’s view altogether. Requirement 10.3.3 asks for audit logs to be backed up promptly to a central log server or other media that is difficult to modify, which is the job the webhook to the SIEM does, and the 12 months of history that Requirement 10.5.1 asks for is the figure to set the topic’s retention or the SIEM’s retention against.

Read encrypted payloads. A company that encrypts card data inside its messages can load its own custom SerDes into Kpow, so authorised users read the decrypted record in Data Inspect while the bytes on the brokers stay encrypted; where the custom SerDes returns JSON, data policies still mask card fields in the decrypted record. TD Bank’s platform team, which began as a payment modernisation project inside the bank’s payments group, describes this arrangement in the same talk. The bank’s network could not connect directly to a Kafka service on the internet and expose its credit card data, so it encrypts every message at message level with a library built for all of its teams, and it loads its own JAR into Kpow so that the people allowed to inspect a topic see the decrypted record.

Watch consumer lag on the payment path. Kpow publishes consumer group offsets and lag, with broker, topic and connector metrics, on Prometheus endpoints for Grafana, AlertManager or the company’s own monitoring, so a team sees an authorization or settlement consumer falling behind before merchants do. On a fraud topic consumer lag is a measure of exposure, which the guide to Kafka monitoring puts as five minutes of lag on a transactions topic meaning five more minutes of unchecked transactions. A bare offset count can mislead, as SoftwareMill’s analysis of lag monitoring sets out, because the same number of messages can be one second or one day of delay, and it recommends watching lag per partition for debugging. Where records are keyed by merchant or card, one partition can fall far behind the rest because it carries one large merchant, so lag is best read per partition as well as per group. Collecting it by hand does not keep up at that level of detail. TD Bank’s platform team says in its talk that gathering consumer lag with command-line tools took 20 minutes or more per cluster and that the same job through the Kpow API runs within two minutes, so the bank collects lag every five minutes and its client teams raise their own tickets from it.

Keep broker ACLs for services. Kpow governs people working through Kpow, not services connecting to brokers, so broker ACLs or an authorizer remain the control for payment applications. The three layers do different jobs: ACLs decide what a client principal can do at the broker, RBAC in the tool decides what a person can do through it, and tenants decide what that person sees. Kpow’s masking hides card data from people and does not change what is stored, so card numbers that should never reach Kafka in the clear still need encryption or tokenisation before they are produced, and its masking is set per topic and not per viewer.

To see these screens before installing anything, the live Kpow demo needs no signup and shows data inspect, consumer groups, topic management and the audit log topic on two MSK clusters. A reader can check the rubric against the product there by working through what a payments engineer does during an incident, searching topic data and checking consumer groups and lag, and then opening the __oprtr_audit_log topic on MSK Secondary to see what the audit trail records. The demo has no SSO and no data policies configured, so sign-in and masking are the two things to test in the company’s own environment. To run the workflows against the company’s own clusters, install Kpow from its container image or JAR; tenants, RBAC, temporary policies, staged mutations, data policies and the audit log are Kpow Enterprise features, and Community Edition is free for 3 clusters and 10 users.

Kpow live demo

Open Kpow the way a payments team would

The live Kpow demo needs no signup. Search topic data, check consumer groups and lag, then read the audit trail on the __oprtr_audit_log topic of the MSK Secondary cluster.

For platform teams running Kafka for card, acquiring and fraud teams.

Try the Kpow demo→

FAQ

What is the best Kafka UI for a payments company?

On this page’s rubric, Kpow: it masks card fields on the server, grants time-boxed production access through its API, records every action and query with the person from the company’s directory, scopes teams with tenants, and runs as one container with no external database and nothing in the data path. Conduktor Console offers more flexible per-viewer masking, but its data-level controls need Gateway, a proxy in front of the brokers.

Does a Kafka management tool fall under PCI DSS?

If it can read topics that carry cardholder data, yes. The PCI SSC’s scoping guidance brings into scope every system that can connect to the cardholder data environment, so the tool, its sign-in and its logs are assessed with it. What differs between tools is how much they add: a self-hosted Kafka client adds one component, while a proxy that producers and consumers connect through also transmits every card message that passes it.

How should engineers debug payment topics without seeing card numbers?

Mask the card fields in every tool engineers use to inspect payment topics, keeping at most the last four digits, and grant production read access per incident rather than permanently. In Kpow that is a data policy using ShowLast4 and a temporary policy, and both the grant and each query appear in the audit log.

How long does Kpow keep its audit log?

Kpow shows the last seven days of the audit log in the product and writes every record to the __oprtr_audit_log topic, whose default retention is one week. Both defaults are shorter than PCI DSS asks for: Requirement 10.5.1, as Microsoft’s guidance on PCI DSS Requirement 10 restates it, is to retain audit log history for at least 12 months with the most recent three months immediately available for analysis. A payments company therefore raises the retention on that topic or sends mutations and data inspect queries to its SIEM through webhooks, which carry the user, the roles and the cluster for each event.

Does masking in a Kafka UI protect the data on the brokers?

No. View-time masking hides values from people inspecting topics through that tool and leaves the stored records and what consumers read unchanged. Card numbers that should never sit in Kafka in the clear need encryption or tokenisation before they are produced; the options are compared in Kafka data masking tools.

Should card numbers be tokenized before they reach Kafka?

Where no consumer needs the full card number, yes: a token or an encrypted field keeps the number out of every topic, every consumer and every tool that reads them, which view-time masking in a UI cannot do. The Council’s tokenization guidelines say that tokenization does not remove the need to maintain and validate PCI DSS compliance but may simplify validation by reducing the number of system components the requirements apply to. Masking in a tool such as Kpow then covers the fields that remain sensitive, such as the last four digits, cardholder names and expiry dates, for the people who inspect topics. The encryption, tokenisation and proxy options are compared in Kafka data masking tools.

Does PCI DSS require a separate Kafka cluster for card data?

No. The PCI SSC’s scoping guidance says network segmentation of the cardholder data environment is not a PCI DSS requirement, and recommends it because it can reduce the scope and cost of the assessment and the difficulty of maintaining the controls. Without adequate segmentation the whole network is in scope. A payments company that keeps card topics on their own clusters in their own network zone therefore does it to shrink the assessment, and a management tool that reaches those clusters is in scope with them, so the instance that manages card clusters is best kept inside the same zone.

Does a Kafka UI need multi-factor authentication under PCI DSS?

If it can show topics that carry cardholder data, yes. Requirement 8.4.2, as Microsoft’s guidance on PCI DSS Requirement 8 restates it, is that multi-factor authentication is implemented for all access into the cardholder data environment. A Kafka tool usually meets it by handing sign-in to the company’s identity provider through SAML or OpenID Connect, where the second factor is enforced, so a tool with no sign-in of its own depends on the proxy in front of it for both factors.

Does Kpow add latency to payment processing?

Not to the payment requests themselves. Kpow connects to the brokers as an ordinary Kafka client beside the payment flow, so producers and consumers never pass through it. It still reads from the brokers like any consumer, and it snapshots each cluster every minute and keeps its own topics on the company’s cluster, which is why its Data Inspect searches stop at a result limit or a scan limit. A tool whose controls work through a proxy is different: every producer and consumer routed through it takes an extra network hop.

How these tools were scored

Six criteria, each taken from a situation payments platform teams meet when card data flows through Kafka, score every option from 0 to 10. They are listed here in order of weight.

1. Out of the data path. A management tool that runs in the company’s own environment, connects as an ordinary Kafka client and keeps no data outside the company’s clusters gives the shortest answer when an assessor asks which components handle card data. Scored lower: tools that need an external database, and tools whose controls work only when application traffic passes through a vendor’s proxy. A self-hosted container with no external database and no proxy scores 9, a tool with a database of its own 6, one with several databases or a component on the brokers 4, and one that needs both a database and a proxy for its controls 3; 10 is kept for an option with nothing to deploy at all. This criterion is scored the same way on every Factor House page that uses it, and only its weight changes with the reader.

2. Production access on request. Can an engineer be given read access to a production topic for one incident, approved and time-boxed, without a standing grant? Can production writes be held for a second person’s approval? PCI DSS Requirement 7 restricts access to cardholder data by business need to know, and a grant that expires on its own is the most direct way to show that need lasted as long as the incident. The wider set of controls over deletes and offset resets is compared in Kafka destructive operations tools.

3. Audit trail per person. When people work through a shared tool, the broker only sees the tool’s service account, so only the tool’s own log can name the person. PCI DSS Requirement 10 asks for access to cardholder data to be logged and monitored, and the PCI SSC glossary describes an audit log as a trail sufficient to reconstruct the events around an operation, so the log has to include data reads as well as changes and be readable without building a consumer first. Kafka audit logging tools compares the layers in detail.

4. Who can see unmasked data. Production access asks whether someone can get into a topic and for how long; this criterion asks what they see once they are in. It scores who can see the real card number through the tool, whether masking can be bypassed, and whether the team that owns the data can be exempted while everyone else sees it masked. The scores are the same as on Kafka data masking tools, which compares masking in more depth, and Control Center, which that page does not score, gets 2 because no masking is described for it. Kafdrop, which that page does not score either, gets 0 because it has neither masking nor sign-in of its own.

5. Directory and Kafka sign-in. PCI DSS Requirement 8 asks for every user to be identified and authenticated, so people should sign in through the company’s identity provider, whether that is SAML, OIDC or LDAP, with directory groups mapped to roles rather than shared logins. On the cluster side the tool has to connect the way the company’s clusters already authenticate clients. Kafka SSO tools covers the protocol detail.

6. Many teams, shared clusters. Acquiring, issuing, fraud and settlement teams on a handful of clusters need each team scoped to its own topics, consumer groups and connectors, so the platform team can onboard a team with a policy rather than a cluster, and nobody sees another team’s card topics by default.

The cost figures model a payments company running 4 clusters (development, test, pre-production and production) for 100 engineers at $120 an engineer-hour, and each card prints its own assumptions. The general listicle view, without the payments weighting, is in best Kafka management tools.

F1 What a payments company runs into, and what the Kafka tool has to do about it

What happens at a payments company What the Kafka tool has to do

Card data in topics Authorization, settlement and dispute topics carry card numbers and cardholder details Mask card fields on the server for everyone who inspects those topics, down to the last four digits

Debugging a failed payment An engineer needs to read one transaction's messages in production to find out why it failed Grant read access on request, time-boxed, through an API a change system can call, and record the grant

Assessment evidence The assessor asks who accessed cardholder data, and when Log every action and data query with the person from the directory, not a shared service account

Directory sign-in People and groups live in the company's identity provider, behind SAML, OIDC or LDAP Sign people in through that directory and map its groups to roles

Many teams Acquiring, issuing, fraud and settlement teams share a small number of clusters Scope each team to its own topics, consumer groups and connectors

The authorization path Where authorization messages travel over Kafka, every extra component between a terminal and an approval adds latency and a point of failure Run beside the brokers as a Kafka client, not in front of them as a proxy that every payment passes through

Every option is scored from 0 to 10 on each criterion, from the evidence and sources this page cites, and the reason for each score is on its card. The criteria are weighted: Out of the data path counts three times, Production access on request counts twice, Audit trail per person counts twice, Who can see unmasked data counts once, Directory and Kafka sign-in counts once and Many teams, shared clusters counts once, for a total out of 100. Out of the data path counts three times because PCI DSS applies to every component that stores, processes or transmits cardholder data, and a proxy that producers and consumers connect through transmits every card message that passes it, which puts it inside the cardholder data environment and, where authorization messages travel over Kafka, on the authorization path; a tool that needs a database of its own is one more component to bring into the assessment. Production access and the per-person audit trail count twice, because restricting access to cardholder data by business need to know, and logging every access to it, are PCI DSS Requirements 7 and 10. Who can see unmasked data, directory sign-in and shared clusters count once. This page is published by Factor House, which makes Kpow. Every option is scored on the same rubric and the same sources: Kpow's per-criterion scores are set the same way as every other option's and are not adjusted, and the weights apply to every option alike. Kpow ranks first on its total of 87 out of 100. The other options follow by total. Conduktor is listed last whatever its total; on its total of 58 it would place third.

Related reading

• Kafka: the complete guide
• Best Kafka governance tools for financial services
• Best Kafka management tools for banks
• Kafka data masking tools
• How PayPal uses Apache Kafka in production
• Kafka audit logging tools
• Best Kafka management tools for crypto exchanges
• Best Kafka management tools for lenders and consumer fintechs