News, guides, and engineering deep dives.
News, guides, and engineering deep dives on Kafka, Flink, Iceberg, and real-time data.
How Barclays uses Apache Kafka in production
A deep-dive into Barclays' Kafka architecture — covering dual-environment deployment on AWS and IBM Z-Linux, operating practices, and the broader streaming stack.
How Cash App uses Apache Kafka in production
A deep-dive into Cash App's Kafka architecture — covering the evently-cloud bridge service, per-topic application-layer encryption, and the internal PubSub platform at Block.
How Datadog uses Apache Kafka in production
A deep-dive into Datadog's Kafka architecture — covering use cases, scale, engineering decisions, and key contributors across hundreds of clusters.
Apache Kafka architecture: a complete guide to internals, components, and deployment
A complete guide to Apache Kafka architecture: internals, components, KRaft, replication, consumers, Connect, Streams, and deployment options.
How Netflix uses Apache Kafka in production
A deep-dive into Netflix's Kafka architecture — covering the Keystone pipeline, Data Mesh platform, scale figures from 700 billion to 2 trillion events per day, and the engineering decisions behind it.
How New Relic uses Apache Kafka in production
A deep-dive into New Relic's Kafka architecture — covering use cases, scale, engineering decisions and key contributors.
How Notion uses Apache Kafka in production
A deep-dive into Notion's Kafka architecture — covering use cases, scale, engineering decisions, and key contributors across their data lake and AI pipelines.
How PagerDuty uses Apache Kafka in production
A deep-dive into PagerDuty's Kafka architecture, covering event ingestion, notification scheduling, task execution, and the engineering decisions behind each.
How Pinterest uses Apache Kafka in production
A deep-dive into Pinterest's Kafka architecture — covering use cases, scale, engineering decisions, and key contributors. From 15 million to 40 million messages per second across 3,000 brokers.