News, guides, and engineering deep dives.
Practical guidance on Kafka, Flink, Iceberg, and real-time data.
Best tools for Kafka audit logging
Kafka audit logging happens at three layers: broker authorization logs, the audit trail of the tools people use, and data lineage. Every option for each layer, scored on the same rubric.
Best tools to manage Kafka broker configs
kafka-configs.sh, Strimzi, Terraform, JulieOps, AKHQ, Kafbat UI, Confluent Control Center and Kpow, scored on seeing the running config, catching drift, changing it safely and knowing who changed it.
Best tools to monitor Kafka broker health
Prometheus with the JMX Exporter, kafka_exporter, KMinion, Cruise Control, Datadog, New Relic, Confluent Control Center and Kpow, scored on the broker signals they actually see.
Kafka CLI commands for day-2 operations
The Kafka CLI commands operators run in production, checked against Kafka 4.3: topics, produce and consume, lag and offset resets, configs, cluster health and ACLs, with KRaft-era flag changes.
Best Kafka CLI tools
Kafka CLIs scored on coverage, contexts, auth, agent safety, maintenance and annual cost: the bundled scripts, kcat, kafkactl, kaf, kcl, rpk, Confluent CLI, Zoe, kt and the Kpow CLI.
Best tools to monitor Kafka Connect connectors
Kafka Connect monitoring tools, from the REST API and JMX to Strimzi, Kpow and Lenses, scored on task state, per-task metrics, restarts, alerting and access.
How to diagnose and fix a failed Kafka Connect connector
A connector reports RUNNING while its tasks have FAILED. The diagnostic sequence, and how to tell a connector fault from a limit in the external system it writes to.
Best tools to monitor Kafka consumer lag
Kafka consumer lag tools compared on one rubric: client-side against cluster-side measurement, offset lag against time lag, per-partition detail and alerting. The CLI, JMX, Burrow, Kpow and more.
How to find and mask PII in Kafka topics
An audit finds card numbers or emails readable in a Kafka topic. How to find every field that carries PII, choose where to mask it, and prove the masking holds, with the commands for each step.