A deep-dive into PayPal's Kafka architecture — covering use cases, scale, engineering decisions, and key contributors across a fleet handling 1.3 trillion messages per day.

A deep-dive into The New York Times' Kafka publishing pipeline — covering the Monolog architecture, single-partition design, Kafka Streams usage, and the engineering decisions behind treating Kafka as a permanent content store.

A deep-dive into Grab's Kafka architecture — how the Coban team built a terabyte-per-hour streaming platform serving 300 billion events a week across GrabFood, GrabPay, mobility, and more.

A deep-dive into Wix's Kafka architecture: 66 billion daily messages, 2,200+ microservices, the Greyhound SDK, Confluent Cloud migration, and operating 500,000+ partitions across 4 regions.

A deep-dive into Pinterest's Kafka architecture — covering use cases, scale, engineering decisions, and key contributors. From 15 million to 40 million messages per second across 3,000 brokers.

A deep-dive into Goldman Sachs's Kafka architecture — covering use cases across three divisions, migration to Amazon MSK, resilience design, and key engineering decisions.

A deep-dive into JPMorgan Chase's Kafka architecture — covering multi-tenant cluster design, managed Kafka Connect, the Photon Framework, and the engineering decisions behind one of the largest financial services deployments.

A deep-dive into New Relic's Kafka architecture — covering use cases, scale, engineering decisions and key contributors.

A deep-dive into PagerDuty's Kafka architecture, covering event ingestion, notification scheduling, task execution, and the engineering decisions behind each.

A deep-dive into Robinhood's Kafka architecture: use cases, scale, engineering decisions, and key contributors. Learn how Robinhood processes 2.2 million messages per second across equities trading, crypto, fraud detection, and more.

A deep-dive into Salesforce's Kafka architecture — covering use cases, scale, engineering decisions and key contributors across a fleet of 100+ clusters processing 3+ trillion events per day.

A deep-dive into Pinterest's Kafka architecture — covering use cases, scale, engineering decisions, and key contributors. From 15 million to 40 million messages per second across 3,000 brokers.

A deep-dive into Reddit's Kafka architecture — covering use cases, scale, engineering decisions and key contributors.

A deep-dive into Spotify's Kafka architecture — covering their event delivery system, 700K events/second scale, engineering decisions, and why they ultimately migrated to Google Cloud Pub/Sub.

A deep-dive into Shopify's Kafka architecture — covering CDC at 100,000 records/sec, Kubernetes deployment, the Sarama Go client library, and BFCM scale engineering.

A deep-dive into Netflix's Kafka architecture — covering the Keystone pipeline, Data Mesh platform, scale figures from 700 billion to 2 trillion events per day, and the engineering decisions behind it.

A deep-dive into Barclays' Kafka architecture — covering dual-environment deployment on AWS and IBM Z-Linux, operating practices, and the broader streaming stack.

A deep-dive into Airbnb's Kafka architecture — covering six production systems, 35+ billion daily events, SpinalTap CDC, Flink-based personalisation, and Kafka as a write-ahead log.

A deep-dive into LinkedIn's Kafka architecture, covering use cases, scale, engineering decisions, and key contributors.

A deep-dive into Cloudflare's Kafka architecture: use cases at trillion-message scale, 14 clusters, internal tooling decisions, and the engineering lessons behind a decade of Kafka operations.

A deep-dive into Uber's Kafka architecture - covering use cases, scale, engineering decisions, and key contributors. From one region to trillions of messages a day.

12 best practices for Kafka data observability covering consumer lag monitoring, schema enforcement, end-to-end auditing, DLQs, and lineage, with an implementation roadmap.

How large should Kafka messages be in production? Covers sizing tiers, the four-config chain, compression codecs, and patterns for handling payloads above 1 MB.

A technical guide to Kafka message key best practices covering partitioning, ordering guarantees, hot keys, log compaction, and serialization for production systems.

Kafka ships insecure by default. Learn how to build a production-ready Kafka security architecture covering TLS encryption, SASL authentication, ACLs, audit logging, and network isolation.

A practical guide to Kafka cluster management: architecture sizing, day-to-day operations, performance tuning, KRaft migration, and monitoring for production clusters.

How Kafka partition keys work, what makes a good key, and practical guidance on cardinality, hot partitions, compaction, cross-language hashing, and safe key migration.

How to size, manage, and operate Kafka topic partitions correctly. Covers the throughput formula, keyed topic risks, KRaft limits, and common anti-patterns.

Compare the 10 best Kafka management tools for 2026, including Kpow, AKHQ, Conduktor, and Confluent Control Center. Covers pricing, RBAC, and deployment requirements.

A practical guide to scaling Apache Kafka in production, covering partitioning strategy, consumer group design, broker sizing, KRaft migration, and more.

Compare 12 Kafka monitoring tools for 2026, from enterprise-grade Kpow to open-source AKHQ and Prometheus. Covers deployment, pricing, and key trade-offs.

This guide covers CDC fundamentals, compares the dominant implementation patterns, walks through a working PostgreSQL-to-Kafka setup, and shows where CDC fits in a broader data mesh architecture.

Kpow 94.5 enhances data inspection with comma-separated kJQ Projection expressions, in-browser search, and flexible deserialization options. This release also adds high-performance streaming for large datasets and expands kJQ with new transforms and functions—testable on our new interactive examples page. These updates provide deeper insights and more granular control over your Kafka data streams.

Kpow now offers enhanced under-replicated partition (URP) detection for more accurate Kafka health monitoring. Our improved calculation correctly identifies URPs even when brokers are offline, providing a true, real-time view of your cluster's fault tolerance. This helps you proactively mitigate risks and ensure data durability.

We're excited to launch the new Factor House Docs, a unified hub for all our product documentation. Discover key improvements like a completely new task-based structure, interactive kJQ examples, and powerful search, all designed to help you find the information you need, faster than ever. Explore the new home for all things Kpow, Flex, Factor Platform and more.

Jumpstart your journey into modern data engineering with Factor House Local. Explore pre-configured Docker environments for Kafka, Flink, Spark, and Iceberg, enhanced with enterprise-grade tools like Kpow and Flex. Our hands-on labs guide you step-by-step, from building your first Kafka client to creating a complete data lakehouse and real-time analytics system. It's the fastest way to learn, prototype, and build sophisticated data platforms.

Kpow's 94.3 release is here, transforming how you work with Kafka. Instantly query topics using plain English with our new AI-powered filtering, automatically decode any message format without manual setup, and leverage powerful new enhancements to our kJQ language. This update makes inspecting Kafka data more intuitive and powerful than ever before.

The real-time ecosystem has outgrown Kafka alone. At Current London 2025, the transition from Kafka Summit was more than a name change — it marked a shift toward streaming-first AI, system-level control, and production-ready Flink. Here's what Factor House saw and learned on the ground.

The EU Data Act takes effect in September 2025, introducing major implications for teams running Kafka. This article explores what the Act means for data streaming engineers, and how Kpow can help ensure compliance — from user data access to audit logging and secure interoperability.

Discover how our 94.1 release has streamlined DockerHub, Helm Charts, and AWS Marketplace deployments!

2025 is a pivotal moment at Factor House (formally Operatr.IO). We've announced our fundraise and have much more to announce about our roadmap this year. This is why we think that now is the perfect time to do a bit of spring cleaning and retire the io.operatr artifacts for good.

At Factor House, delivering reliable software is at the heart of everything we do. A key aspect of this commitment lies in our approach to managing Java compatibility. This blog post outlines our current release process and future plans for evolving Java support, including our approach to deprecating older versions in a way that respects the needs of diverse customer bases.

We first published a VPAT in the release notes of Kpow for Apache Kafka v92.4, with a VPAT available to download in every release of Kpow since. Today, we are pleased to announce that we are extending that commitment to all future Factor House product releases - including Flex for Apache Flink and the Factor Platform.

With our new API, you can now leverage Kpow's capabilities directly from your own tools and platforms, opening up a whole new range of possibilities for integrating Kpow into your existing workflows. Whether you're managing topics, consumer groups, or monitoring Kafka clusters, our API provides a seamless experience that mirrors the functionality of our user interface.

Today we introduce Flex for Apache Flink and announce Factor Platform, the future of distributed systems engineering. Starting now, individuals can use Flex CE for free, even at work. Organisations can install Flex CE in up to three non-production environments.