News, guides, and engineering deep dives.
Practical guidance on Kafka, Flink, Iceberg, and real-time data.
Industry Apache Kafka 4.3.0: A guide for platform engineers
Kafka 4.3.0 covers broker cordoning, partition size metrics, share group tuning, and tiered storage fixes. Here's what platform engineers need to act on.
Product Data lineage support in Factor Platform
Learn how Factor Platform brings OpenLineage metadata into your Kafka environment, making data ownership, PII classification, and lineage visible by default.
Industry What the IBM-Confluent deal means for Kafka users
IBM's $11B Confluent acquisition raises questions for Kafka users. Assess your lock-in risk across Schema Registry, managed connectors, and operational tooling.
Best tools to monitor and operate Flink jobs
Flex ranks first of five tools for monitoring and operating Apache Flink jobs, scored on seeing a job's state, acting on it, alerting, controlling access and staying out of the data path.
Best Flink UI for submitting and managing jobs day to day
Flex and Ververica Platform tie on 81 of 100 among four Flink UIs for daily job work, scored on submitting, stopping and restoring jobs, inspecting them, and reading exceptions and logs.
Best tool to manage Kafka and Flink together: six options scored
Kpow and Flex rank first of six ways to manage Apache Kafka and Apache Flink together, scored on the data path, one access model, approvals, audit, Flink jobs and sign-in.
Best management tools for teams running Kafka to Iceberg pipelines
Kpow ranks first of seven Kafka and Flink tools for running a Kafka to Iceberg pipeline, scored on the data path, Connect sink tasks, sink lag, Flink checkpoints, access and audit.
How to find the bottleneck behind Flink backpressure
A Flink job is backpressured, sources slow down, latency grows and checkpoints time out. How to find the operator that causes it, tell a slow sink, skewed keys, a slow external call and too little parallelism apart, and fix the right one.
How to fix Flink checkpoints that fail or time out
A Flink job keeps restarting or falling behind because checkpoints fail, expire or take longer every hour. How to find the stage at fault, the metric that proves it, and the one config change to try first.
How to fix a Flink job stuck in a restart loop
A Flink job keeps cycling between RUNNING and RESTARTING. How to find the root exception, tell a transient failure from a permanent one, and set a restart strategy that stops the loop.
How to recover from a Flink JobManager failure with HA
The Flink JobManager pod was evicted, its node was lost or it lost leadership, and jobs restarted, looped or disappeared. What JobManager high availability protects, how to confirm it is configured and working, and how to fix a failover that loops or a cluster that did not recover.
How to fix Kafka consumer lag that looks wrong for a Flink job
Kafka consumer group lag for a Flink job keeps growing, stays flat, jumps after a restart or shows no committed offsets. Why the Flink Kafka source commits offsets only on checkpoints, how to read the Flink-side lag metrics, and what to change.