News, guides, and engineering deep dives.
Practical guidance on Kafka, Flink, Iceberg, and real-time data.
How to fix Flink windows that never fire or drop late events
An event-time Flink job emits no window output or silently drops late events. How to find the cause from the watermark metrics and fix idleness, timestamps, bounds and lateness.
Iceberg writer and reader see different tables
A Kafka to Iceberg writer commits, but a reader sees no data or fails with table not found. Find which catalog, warehouse or namespace each side resolves, then fix it without losing rows.
Iceberg commits failing or conflicting: find and fix the cause
A streaming writer stops committing to an Iceberg table with CommitFailedException or exhausted retries. How optimistic concurrency works, how to find the conflicting job, and how to fix it.
Iceberg storage keeps growing: expire snapshots and orphan files
A streaming-fed Iceberg table keeps growing because every commit adds a snapshot and old files stay referenced. How to measure it, then expire snapshots and remove orphan files safely.
Kafka to Iceberg pipeline breaks after a schema change
A producer changes a field and the Kafka to Iceberg sink fails or drops data. What Iceberg schema evolution allows, what the sink does with an unknown field, and how to fix it.
Kafka to Iceberg: too many small files, and how to compact them
A Kafka to Iceberg pipeline writes many tiny data files on every commit, so query planning slows and metadata grows. Why it happens, how to measure it, and how to fix it with compaction and sink settings.
Best Kafka management tools for transport and logistics companies
Kpow ranks first of six Kafka management tools for transport and logistics companies such as Schneider, Knight-Swift, US Foods and 90POE, scored on the data path, search, clusters and access.
Best Kafka management tools for sports betting and iGaming operators
Kpow, used by FanDuel, ranks first of six Kafka management tools for sports betting and iGaming operators, scored on the data path, production access, audit, masking and search.
Best Kafka management tools for technology and SaaS companies
Kpow ranks first of six Kafka tools for technology and SaaS companies, scored on the data path, multi-cluster reach and monitoring. HPE, ServiceNow, Garmin, TripleLift and JobNimbus run Kpow.