Kpow v90.1 is packed with new Connect and Data Inspect features, provides a refreshed UI, and introduces Kpow Community Edition (CE).

Kpow Community

Just quietly, we have soft-launched Kpow Community Edition (CE) - the new, free version of Kpow.

Start Kpow CE with no config and a wizard UI will guide you through setting up cluster, schema, and connect resources.

Kpow CE will integrate one Kafka Cluster, one Schema Registry, one Connect Cluster, with the UI supporting one user at a time.

Kpow CE contains all the normal Kpow features that engineers love, without any enterprise features (e.g. Auth, RBAC, etc).

Install as many instances as you need, in any environment you like. No sign-up of license required, just pull the docker container and go.

Get started with Kpow Community, or view the Kpow Feature Matrix to see how Community differs from normal Kpow.

Further information will be provided when we officially launch Kpow CE, next week.

Kpow UI Refresh

Kpow v90.1 introduces a complete UI overhaul - we think it looks pretty good!

Kafka Connect Improvements

Kpow v90.1 has a raft of new Kafka Connect improvements.

Improved Debezium Connector Integration

Kpow will now correctly infer topics for Debezium connectors, allowing us to show fuller data on sink and source data consumption and production.

Additional Data in Connect Tables

Kpow now shows last-activity measurements for connectors, along with data volume measurements.

Unreachable Connectors

Occasionally connectors appear in the list provided by your Connect installation, but when we query them individually they are unavailable - often this is related to underlying issues with our resources related to the connector, for example a MySQL sink where the MySQL DB is down.

Previously these connectors would simply not show in the Kpow UI, from v90.1 we highlight these connectors as 'Unreachable', provide the stacktrace that we receive with information about their failed state, and allow you to take actions like restarting the connector.

Create Connector Improvements

The new Kpow UI features a much improved UX for creating new connectors - this looks particularly impressive when integrating Confluent Managed Connect, as they have a lot of connectors to display!

Topic Truncate

The Topic and Topic Partition details tables now offer a 'Truncate Topic' option in the action menu.

With this action you can truncate data in topics up to a particular offset, removing all records if desired.

Kpow offers the ability to truncate to specific offset, to truncate entirely, or to truncate to the current offset of a particular consumer group - meaning you can easily drop messages that are known to have been consumed.

Keep an eye out for next weeks feature blog-post on deleting, truncating, and tombstoning data with Kpow.

Data Tombstoning

Kpow v90.1 intrudoces the ability to explicitly tombstone Key and/or Value fields when producing data.

Simply don't select a serde for key or value and a nil field will be written. This has the impact of deleting data in compacted topics and ktables.

Keep an eye out for next weeks feature blog-post on deleting, truncating, and tombstoning data with Kpow.

Kpow v98.4 introduces a new CONNECT_INSPECT RBAC permission and fixes a number of minor UI bugs.

CONNECT_INSPECT

A new 'View Config' action allows users to view connector configuration without entering the full connector edit UI.

This action is guarded by a new RBAC permission, CONNECT_INSPECT.

Provide users this permission and deny CONNECT_EDIT to allow users to view connector configuration without allowing them to edit connectors.

Users who have CONNECT_EDIT are considered to automatically also have CONNECT_INSPECT (editing a connector means you can see its configuration).

Set ALLOW_CONNECT_INSPECT to apply this new permission when using global access controls, or add CONNECT_INSPECT to your existing RBAC configuration as you would any other permission.

‍

Kpow v89.3 introduces the ability to disable the log.cleaner.enable=true broker configuration check that Kpow requires by default.

What is Log Cleaner Enable?

log.cleaner.enable is a Kafka broker configuration setting that governs data deletion, it is true by default.

When the log cleaner is enabled your Kafka brokers will periodically delete TTL'd data. If you change this configuration to false then no data will ever be physically deleted from your cluster, regardless of whether it has expired or not.

Why Does Kpow Require Log Cleaner Enable?

Kpow writes data into internal topics in your primary cluster, this data helps Kpow perform long-term analysis on the health of your clusters. We drastically reduced the amount of data written to internal topics in a recent release, but Kpow still relies on TTLd data being periodically deleted from your cluster.

Given a long enough period of operation without any data deletion Kpow's internal topics could grow to a considerable size. Being conservative, previous releases of Kpow will refuse to start if the brokers of your primary cluster are not configured to delete data.

How Do I Disable the Log Cleaner Check?

Update to v89.3 and start Kpow with the new environment variable setting: DISABLE_LOG_CLEANER_CHECK=true.

Is It Safe to Disable the Log Cleaner Check?

In the short-term, we believe so. Kpow has significantly reduced the amount of data written to internal topics such that this has become a less significant issue. The next major release of Kpow (v90) has an ephemeral mode where you can choose to run Kpow without writing any data to internal topics.

From v90, Kpow will detect where your primary cluster does not support data deletion and automatically switch into ephemeral mode.

Kpow v89.2 resolves minor issues with XML messages in Data Inspect and relay state when re-authenticating SAML sessions.

Kpow v89.1 is a major feature release that includes up to 20x improvement in Data Inspect speed, support for Confluent Managed Connect, support for MSK Managed Connect, and support for MSK Serverless. This release also includes several minor UI/UX bug fixes and optional basic authentication for Prometheus egress.

Data Inspect Performance

Improving the speed of Kpow's Data Inspection function was a priority for our team through this release. We incorporated feedback from customers and built a test corpus spanning multiple message types, sizes, and compression formats. We will provide a deep dive blog post on these improvements, but the long and short of it is that you can expect a search speed-up of between 3x to 20x depending on those topic factors.

Managed Connect

Kpow's Kafka Connect functionality is extended to support both Confluent Managed Connect, and MSK Managed Connect.

• Confluent Managed Connect Documentation
• MSK Managed Connect Documentation

MSK Serverless

Kpow supports integration with MSK Serverless, see our documentation for configuration and limitations.

• MSK Serverless Documentation

Prometheus Basic Authentication

Kpow provides optional Basic Authentication for Prometheus endpoints, see our documentation for configuration.

• Prometheus Authentication Documentation

Kpow v88.7 resolves a minor issue introduced in Kafka 3.2.0, read the bug breakdown here.