News, guides, and engineering deep dives.
Practical guidance on Kafka, Flink, Iceberg, and real-time data.
How Pinterest uses Apache Kafka in production
A deep-dive into Pinterest's Kafka architecture, covering use cases, scale, engineering decisions, and key contributors. From 15 million to 40 million messages per second across 3,000 brokers.
How Salesforce uses Apache Kafka in production
A deep-dive into Salesforce's Kafka architecture, covering use cases, scale, engineering decisions and key contributors across a fleet of 100+ clusters processing 3+ trillion events per day.
How Shopify uses Apache Kafka in production
A deep-dive into Shopify's Kafka architecture, covering CDC at 100,000 records/sec, Kubernetes deployment, the Sarama Go client library, and BFCM scale engineering.
How Tencent uses Apache Kafka in production
A deep-dive into Tencent's Kafka architecture, covering their federated cluster design, 20 trillion messages per day, KIP contributions, and tiered storage at Tencent Cloud.
How Wix uses Apache Kafka in production
A deep-dive into Wix's Kafka architecture: 66 billion daily messages, 2,200+ microservices, the Greyhound SDK, Confluent Cloud migration, and operating 500,000+ partitions across 4 regions.
How Airbnb uses Apache Kafka in production
A deep-dive into Airbnb's Kafka architecture, covering six production systems, 35+ billion daily events, SpinalTap CDC, Flink-based personalisation, and Kafka as a write-ahead log.
How Bytedance uses Apache Kafka in production
ByteDance ran Kafka at tens of TB/s before replacing it with ByteMQ, a Kafka-compatible platform separating storage from compute. How it works, and why migrating cut resource cost by roughly 70%.
How Cloudflare uses Apache Kafka in production
A deep-dive into Cloudflare's Kafka architecture: use cases at trillion-message scale, 14 clusters, internal tooling decisions, and the engineering lessons behind a decade of Kafka operations.
How DoorDash uses Apache Kafka in production
A deep dive into DoorDash's Kafka architecture, covering the Iguazu event platform, Flink-based ML feature pipelines, self-serve topic governance, and hundreds of billions of daily events.