Kpow v91.1 introduces a new ksqlDB UI, new disk usage telemetry, schema registry improvements, wildcard filtering, and closes a number of minor issues.
ksqlDB User Interface
Kpow now provides ksqlDB integration paired with Kpow's enterprise features such as role-based access control and data-masking.
ksqlDB features seamlessly integrate with other Kpow managed resources such as connect clusters and schema registries.
Like all actions performed within Kpow, an audit log is kept for data governance when issuing any ksqlDB action.
See the ksqlDB setup guide for configuration details, or read our ksqlDB feature guide for more information about Kpow's new ksqlDB capabilities.
Broker Disk Telemetry
Kpow v91.1 provides deeper insight into broker disk usage, all the way down to a directory level. Simply navigate to Brokers > Disk.
Where your Kafka brokers are v3.3.0+ you will see Total Disk, Usable Disk, and Disk Remaining % information.
Broker Disk Telemetry in Prometheus
Each of these new fields is available via Kpow's prometheus endpoint at a broker and broker directory level.
Broker total and usable bytes
# HELP broker_bytes_total The total disk size in bytes of a broker
# TYPE broker_bytes_total gauge
broker_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",} 1.05491914752E11
broker_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",} 1.05491914752E11
broker_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",} 1.05491914752E11
# HELP broker_bytes_usable The usable disk size in bytes of a broker
# TYPE broker_bytes_usable gauge
broker_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",} 1.00089417728E11
broker_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",} 1.00089393152E11
broker_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",} 1.00089397248E11
Broker directory total and usable bytes
# HELP broker_dir_bytes_total The total disk size in bytes of a broker directory
# TYPE broker_dir_bytes_total gauge
broker_dir_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",dir="_kafka_datalogs_logs",} 1.05491914752E11
broker_dir_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",dir="_kafka_datalogs_logs",} 1.05491914752E11
broker_dir_bytes_total{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",dir="_kafka_datalogs_logs",} 1.05491914752E11
# HELP broker_dir_bytes_usable The usable disk size in bytes of a broker directory
# TYPE broker_dir_bytes_usable gauge
broker_dir_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="3",dir="_kafka_datalogs_logs",} 1.00089417728E11
broker_dir_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="2",dir="_kafka_datalogs_logs",} 1.00089397248E11
broker_dir_bytes_usable{domain="cluster",id="CuxsifYVRhSRX6iLTbANWQ",target="1",dir="_kafka_datalogs_logs",} 1.00089393152E11
Schema Registry Improvements
Kpow v91.1 introduces improvements to schema registry authentication and observation.
Confluent Cloud Schema Registry OAuth Authentication
Kpow supports authenticating with Confluent Cloud schema registry via OAuth.
See the Confluent Cloud guide to Schema Registry OAuth and the Kpow Schema Registry Configuration Guide for more information.
Confluent Schema Registry Observation Version
Kpow provides a configurable OBSERVATION_VERSION that allows you to choose a more efficient mode of observering Confluent schema registry, at the small cost of showing schema compatibility information only at the drill-down level in the Kpow UI.
In time we will default the OBSERVATION_VERSION to '2', the current default is '1' (no change from existing).
Read more in the Kpow Schema Registry Configuration Guide.
Custom Serdes and Message Headers
Message Headers are now passed to the serialize and deserialize methods of custom serdes configured with Kpow.
See the Kpow Custom Serdes Guide for more information.
Note: this change introduces a minor breaking change in that your custom serdes must be compiled with Kafka v2.1.0+.