Release v92.2 introduces new UI for Kafka Quotas, Transactional Producers, and KRaft clusters along with new features for topic and consumer management, and improvements to Flink checkpointing and watermarking in Flex.

This is the first multi-product release by Factor House, you will notice that our Changelog now provides entries for Kpow, Flex, and 'All Products'.

Quota Management

Kpow's new Quota UI allows you to view, edit, create, and delete Kafka Quotas.

Mutation of quotas is controlled by a new QUOTA_EDIT RBAC permission.

See our quota documentation for more information.

Transactional Producer Management

Kpow provides a complete management interface to Kafka Producers. This allows a user to fence a producer or abort any (possibly hanging) transactions.

Mutation of producers is controlled by a new PRODUCER_EDIT RBAC permission.

See our producer documentation for more information.

KRaft Management

Kpow's new KRaft UI allows you to view KRaft information and unregister brokers.

Unregistering of brokers is controlled by a new BROKER_UNREGISTER RBAC permission.

See our broker documentation for more information.

Topic Leader Election and Alter Replicas

Release v92.2 introduces support for topic leader election and topic alter replicas.

Topic leader election is controlled by a new TOPIC_ELECT_LEADER RBAC permission, TOPIC_ALTER_REPLICAS controls access to the topic alter replicas function.

See our topic replica documentation for more information.

Consumer Group Management

If you have a static consumer group (a consumer group with group.instance.id set) you can use Kpow to remove a member from the group.

This is can be useful as static consumer groups do not send a leave group request when they go offline, and instead rely on session.timeout to trigger a group rebalance.

See our consumer group member documentation for more information.

Topic Client Monitoring

With the introduction of transactional producer management, Kpow now presents more information about topics under a new 'Clients' tab.

This new Client monitoring allows you to view all transactional producer and consumer groups that are connected to a single topic or topics.

Optional RBAC_EVALUATION_STRATEGY

Kpow now allows you to define an RBAC_EVALUATION_STRATEGY environment variable to allow more lenient evaluation of STAGE permissions.

Kpow is naturally conservative in deciding when a user can take an action on resource. By default this extends to setting a users access to STAGE where the user has both STAGE and ALLOW permission defined to a specific resource.

When RBAC_EVALUATION_STRATEGY=STAGE_LENIENT is set a user will set a users access to ALLOW where the user has both permissions defined.

In every case if a user has DENY or no permission set then they are denied access to the action.

Kpow v92.1 is the 112th release of Kpow and is the first of the v92 major version.

This release closes a large number of support tickets, introduces new features, and changes the default observation behaviour of Kafka Connect and Schema Registry to be more efficient. Consider this release a spring-clean that lays the groundwork for some exciting product news, more to follow shortly.

Default Observation Change

In Kpow v91.1 we introduced a new obervation method for Schema Registry, similarly in Kpow v91.4 we introduced a new obervation method for Kafka Connect.

These new observation methods reduce the number of network calls that Kpow makes to snapshot Schema and Connect resources. In this release we make the new observation methods the default behaviour of Kpow. If required you can revert to the previous behaviour, check the Schema and Connect docs for details.

Time Since URP

Kpow already shows you the details of any Under Replicated Partitions (URP), in this release we add new information to the table of URP data - the time elapsed since the URP was first detected. This elapsed time is useful context when determining the cause and criticality of Under Replciated Partitions.

Multi Topic Create

Kpow v92.1 introduces new UX that allows for the creation of multiple topics at the same time, directly from the Create Topic UI.

Flex for Apache Flink

In anticipation of our GA release of Flex for Apache Flink, containers are deployed to Dockerhub.

Kpow v91.5 brings improved UX for expanding and exploring tabular data, provides support for importing messages with headers for production, and fixes a few minor bugs.

Kpow v91.1 introduces a new ksqlDB UI, new disk usage telemetry, schema registry improvements, wildcard filtering, and closes a number of minor issues.

ksqlDB User Interface

Kpow now provides ksqlDB integration paired with Kpow's enterprise features such as role-based access control and data-masking.

ksqlDB features seamlessly integrate with other Kpow managed resources such as connect clusters and schema registries.

Like all actions performed within Kpow, an audit log is kept for data governance when issuing any ksqlDB action.

See the ksqlDB setup guide for configuration details, or read our ksqlDB feature guide for more information about Kpow's new ksqlDB capabilities.

Broker Disk Telemetry

Kpow v91.1 provides deeper insight into broker disk usage, all the way down to a directory level. Simply navigate to Brokers > Disk.

Where your Kafka brokers are v3.3.0+ you will see Total Disk, Usable Disk, and Disk Remaining % information.

Broker Disk Telemetry in Prometheus

Each of these new fields is available via Kpow's prometheus endpoint at a broker and broker directory level.

Broker total and usable bytes

# HELP broker_bytes_total The total disk size in bytes of a broker# TYPE broker_bytes_total gaugebroker_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",} 1.05491914752E11broker_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",} 1.05491914752E11broker_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",} 1.05491914752E11# HELP broker_bytes_usable The usable disk size in bytes of a broker# TYPE broker_bytes_usable gaugebroker_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",} 1.00089417728E11broker_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",} 1.00089393152E11broker_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",} 1.00089397248E11

Broker directory total and usable bytes

# HELP broker_dir_bytes_total The total disk size in bytes of a broker directory# TYPE broker_dir_bytes_total gaugebroker_dir_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",dir="_kafka_datalogs_logs",} 1.05491914752E11broker_dir_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",dir="_kafka_datalogs_logs",} 1.05491914752E11broker_dir_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",dir="_kafka_datalogs_logs",} 1.05491914752E11# HELP broker_dir_bytes_usable The usable disk size in bytes of a broker directory# TYPE broker_dir_bytes_usable gaugebroker_dir_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",dir="_kafka_datalogs_logs",} 1.00089417728E11broker_dir_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",dir="_kafka_datalogs_logs",} 1.00089397248E11broker_dir_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",dir="_kafka_datalogs_logs",} 1.00089393152E11

Schema Registry Improvements

Kpow v91.1 introduces improvements to schema registry authentication and observation.

Confluent Cloud Schema Registry OAuth Authentication

Kpow supports authenticating with Confluent Cloud schema registry via OAuth.

See the Confluent Cloud guide to Schema Registry OAuth and the Kpow Schema Registry Configuration Guide for more information.

Confluent Schema Registry Observation Version

Kpow provides a configurable OBSERVATION_VERSION that allows you to choose a more efficient mode of observering Confluent schema registry, at the small cost of showing schema compatibility information only at the drill-down level in the Kpow UI.

In time we will default the OBSERVATION_VERSION to '2', the current default is '1' (no change from existing).

Read more in the Kpow Schema Registry Configuration Guide.

Custom Serdes and Message Headers

Message Headers are now passed to the serialize and deserialize methods of custom serdes configured with Kpow.

See the Kpow Custom Serdes Guide for more information.

Note: this change introduces a minor breaking change in that your custom serdes must be compiled with Kafka v2.1.0+.

Kpow v91.4 introduces ARM64 docker builds to support Graviton deployments and includes a range of general feature improvements.

Kpow v91.3 is a minor release that introduces new Schema and Connect features and resolves a number of small issues.

Improved Schema Configurability

This release introduces new configuration options for Schema Registry, including:

• New environment variables when connecting to Schema Registry with Oauth2
• AWS Glue static credentials for authentication
• AWS Glue cross-account AWS access (STS Assume Role support)

See our Schema Registry configuration documentation for more.

Improved Prometheus Reporting

Kpow now provides richer prometheus metrics for consumer groups:

/metrics/v1

The existing metrics endpoint now returns group.member.count and group.host.count gauges for each consumer group

/group-offsets/v1

This new endpoint returns information about every individual group member assignment, including offset, offset-delta, last-read, and first-observed.

Improved Connector Creation

You can now create a connector in Kpow from JSON input, the same as you would post to the Connect REST endpoint.