Chad Harris
- Using Apache Kafka since 0.8.0 (2012)
- 18 years across engineering, architecture and engineering leadership
- 6+ years at Square / Block, Engineering Manager to Engineering Leader
- PCI-compliant, high-volume transactional platforms
In short
Chad Harris is a Solutions Architect at Factor House who has worked with Apache Kafka since 2012 and previously led engineering on PCI-compliant payments systems at Square (Block).
Chad Harris is a Solutions Architect at Factor House, bringing 18 years of experience across software engineering, application architecture, and engineering leadership. He has deep hands-on expertise with Apache Kafka, high-volume transactional systems, and PCI-compliant architectures, most recently as an Engineering Leader at Block (formerly Square). At Factor House, Chad works directly with global enterprise customers to help them improve how they manage, govern, and observe their real-time data.
Expertise
Chad's expertise spans distributed systems, real-time data infrastructure, and enterprise application architecture. He has over six years of production experience with Apache Kafka and extensive background in NoSQL data stores including DynamoDB and Cassandra. His specialisation includes building highly scalable, tokenised security systems and high-volume transactional platforms built to PCI compliance standards. Beyond the technical, Chad is a seasoned engineering leader with a track record of mentoring teams and applying agile and lean methodologies in ways that deliver practical business outcomes.
Experience
Chad is currently a Solutions Architect at Factor House. Before that, he spent over six years at Square (now Block), progressing from Engineering Manager to Engineering Leader. Prior to Square, he was Head of Engineering at Verrency, a payments technology company, where he led engineering across a high-security, high-availability fintech platform. Earlier roles include Technical Lead and Senior Software Engineer positions at Reece Australia, IOOF Holdings, National Australia Bank, and AIA, as well as consulting engagements across financial services and government.
Education
- University of Newcastle
Talks, appearances and mentions
Work published, hosted or co-presented by someone other than Factor House.
- Talk June 10, 2026 · Factor House and Aiven
Things that go bump in the night: Kafka operational issues
co-presented with Hugh Evans, Aiven
Real-world Kafka operational failures, from subtle misconfigurations to full-scale incidents, and the debugging workflows that help. Co-presented with Aiven, whose audience the session was run for.
- Video August 14, 2025 · Factor House
Chad Harris on Factor House's software
recorded while at Block, as a customer
Recorded while Chad was at Block, before he joined Factor House: a practitioner's account of running the tooling as a customer.
Writing and talks for Factor House
- Article August 19, 2026 · Factor House
Dead letter queues in Kafka: a consumer-side, Kafka-only approach
The DLQ pattern for plain Kafka consumers: Kafka-only topics scoped per consumer group, replay through a retry topic rather than the main topic, and why using retries to ride out a downstream outage is the wrong move.
- Article June 27, 2026 · Factor House
Kafka UI: The Ultimate Guide
A 27-minute guide to what a Kafka UI is for, and what operators actually need visual control over: topics, consumers, brokers and connectors.
- Talk June 25, 2026 · Factor House webinar
Kafka operational issues and how to survive them
Four real production incidents, walked through end to end: a consumer group with tens of thousands of idle members overwhelming its coordinator, a config change that silently failed to roll back and filled a broker's disk six months later, a partition increase that left messages unread, and a 20-minute poll interval that turned one stuck batch into a 2am page.
The recording of the 25 June run. The same talk also ran on 29 April, and on 10 June with Aiven (both listed above).
- Video May 29, 2026 · Factor House
Data governance for Apache Kafka: lineage support in Factor Platform
A walkthrough of the OpenLineage metadata support for Apache Kafka in Factor Platform, presented to camera.
- Article May 29, 2026 · Factor House
Data governance for Kafka: introducing lineage support in Factor Platform
How OpenLineage metadata makes data ownership, PII classification and lineage visible by default in a Kafka environment.
- Talk April 29, 2026 · Factor House webinar
Kafka production failures, live tech talk
The first run of the Kafka production failures talk, delivered for a North American audience.
Registration page for a session that has ended.
- Post April 8, 2026 · LinkedIn
On StreamNative forking Kafka: why the Ursa engine changes the math
50 reactions, 2 comments, 3 reposts
"I always liked the idea of Pulsar and the Ursa engine. But there was just one problem with it for me: it wasn't Kafka." On why competing with the Kafka ecosystem is a Sisyphean task, and what changes once a challenger implements the Kafka protocol instead.
- Post March 27, 2026 · LinkedIn
Four questions to ask your platform team after the IBM–Confluent deal
80 reactions, 8 comments, 7 reposts
"You don't need to migrate anything today. But you should know your exposure." Four questions on Confluent-specific features versus standard Kafka APIs, what would break on a move to MSK, Redpanda or Aiven, and how much CI/CD and monitoring is coupled to Confluent tooling.
- Article March 26, 2026 · Factor House
What the IBM Confluent acquisition means for Kafka users
Assessing lock-in risk across Schema Registry, managed connectors and operational tooling after IBM's $11B acquisition.
Chad also posts on LinkedIn.
Latest articles
Apache Kafka vs Confluent Kafka
The broker is the same. What differs is licensing, bundled components, deployment models and support. Where the Apache line sits, what Confluent adds, and the dependency questions to ask.
August 29, 2026Kafka-compatible cloud brokers
Redpanda and the Kafka-compatible brokers, evaluated from the operator's seat: compatibility proof, operational claims, TCO against tiered-storage Kafka, and benchmarks.
August 29, 2026Confluent Kafka in Docker
Running Confluent's Kafka images and the official apache/kafka image in Docker: a working compose file, the advertised-listeners fix, log levels and custom Connect images.
August 29, 2026Data governance policy examples for streaming platforms
Worked data governance policy examples for Kafka: schema compatibility rules, topic lifecycle, PII classification tiers, ACL and RBAC templates, and lineage requirements.
August 29, 2026Debezium vs Kafka Connect
Debezium and Kafka Connect are not alternatives: Debezium is a CDC connector family that runs on Connect. The real decisions are log-based CDC vs JDBC polling, and Connect cluster vs Debezium Server.
August 29, 2026Kafka deployment automation
Deployment automation for Kafka: zero-downtime rolling changes, declarative topics and ACLs, Kubernetes operators, Terraform and GitOps for cluster state.
August 29, 2026The difference between Kafka and RabbitMQ
The difference between Kafka and RabbitMQ is the data model: a replayable log against a delete-on-acknowledge queue. Storage, routing, scaling and when to pick which.
August 29, 2026How RabbitMQ works
How RabbitMQ works, mapped to Kafka concepts: exchanges and bindings, per-message acknowledgement, competing consumers, quorum queues and dead lettering.
August 29, 2026Kafka ACL
A Kafka ACL allows or denies a principal an operation on a resource from a host. Syntax, copy-paste commands, production best practices and GitOps automation.
August 29, 2026Kafka authentication
Kafka authentication verifies every client and broker connection with SASL or mutual TLS. Listener and JAAS blueprints, mechanism choice, rotation and troubleshooting.
August 29, 2026Kafka consumer groups
A Kafka consumer group shares the work of consuming a topic, one partition per member. Troubleshooting lag and rebalances, the CLI cheat sheet, coordinator architecture and offset commits.
August 29, 2026Kafka Connect
Kafka Connect moves data between Kafka and external systems through source and sink connectors. What a deployment is made of, the operational tasks, and where the sub-pages go deeper.
August 29, 2026Kafka Connect MongoDB example
A production Kafka Connect MongoDB example in both directions: source connector via change streams, sink connector with idempotent writes, DLQ settings and secrets handled properly.
August 29, 2026Kafka Connect pricing
Kafka Connect is free open source software. The cost is in running it: managed per-task and per-GB billing, network surcharges, or the engineering hours of self-hosting. The TCO math, honestly.
August 29, 2026Kafka Tool download
Kafka Tool is the former name of Offset Explorer. What each download actually installs, the production jobs a Kafka GUI has to do, and the enterprise constraints to check before connecting one.
August 29, 2026Kafka vs other brokers
How Kafka compares with RabbitMQ and other message brokers: retention models, routing, use cases and operational trade-offs, from the operator's seat.
August 29, 2026Kafka vs RabbitMQ performance
Kafka and RabbitMQ perform differently because their storage models differ. Throughput and latency behaviour, durability trade-offs, benchmark methodology and the workload facts that decide the fit.
August 29, 2026Managed vs unmanaged databases
Managed vs unmanaged databases for teams running Kafka: operational overhead, SLAs, cost architecture, connector and CDC integration, control and security.
August 29, 2026Multi-tenant architecture
A multi-tenant Kafka architecture shares one cluster across teams with quotas, ACLs and naming conventions. Isolation, namespaces, chargeback and topology.
August 29, 2026RBAC roles
RBAC roles bundle permissions into named sets like viewer, operator and admin. How role definitions, resource patterns and operation mappings work across the Kafka ecosystem.
August 29, 2026Kafka stream governance
Stream governance applies data governance to data in motion: schemas enforced at produce time, lineage across topics and jobs, catalogs, and access and quality rules on live streams.
August 29, 2026What is a data governance policy?
A data governance policy is an enforceable rule for how data is structured, accessed, retained and traced. On Kafka it is implemented as configuration and code, not documents.
August 29, 2026What is envelope encryption?
Envelope encryption encrypts data with a local data key, then wraps that key with a KMS-held key. On Kafka it is the pattern that makes per-field encryption work at full throughput.
August 29, 2026What is Kafka Connect?
Kafka Connect streams data between Kafka and other systems as managed, fault-tolerant tasks. The internal architecture, production scaling, error handling and custom development.
August 29, 2026What is Kafka rebalancing?
Kafka rebalancing redistributes a consumer group's partitions when membership changes. The triggers, the three timeouts, cooperative rebalancing, KIP-848 and the metrics that explain incidents.
August 29, 2026What is RabbitMQ?
RabbitMQ explained for Kafka operators: the AMQP queue model, broker-side routing, push delivery, streams, and where it fits beside a Kafka deployment.
August 29, 2026What is Redpanda?
Redpanda reimplements the Kafka wire protocol in a single C++ binary. The architecture, the drop-in compatibility boundaries, and how to evaluate it against tiered-storage Kafka.
August 29, 2026Kafka: The Complete Guide
Apache Kafka is a distributed event streaming platform that stores ordered, replayable records in partitioned topics. This hub covers Kafka fundamentals, operations, governance and tooling.
August 21, 2026Kafka brokers in production
A Kafka broker is the server that stores partition logs and serves produce and fetch traffic. Configuration, troubleshooting, JMX metrics, maintenance and network tuning for production.
August 21, 2026Kafka consumers in production
A Kafka consumer reads records from topic partitions, tracking its own offset. The configuration that decides message loss, rebalance troubleshooting, and poll-loop patterns that survive production.
August 21, 2026Kafka in Docker
Running Kafka in Docker: the official images, reliable Docker Compose topologies, the advertised.listeners trap that breaks local connections, and why a single-node container is not a deployment.
August 21, 2026Kafka offsets
A Kafka offset is a record's position in its partition, and a committed offset is a consumer group's bookmark. Reading lag from CURRENT-OFFSET and LOG-END-OFFSET, plus the reset strategies.
August 21, 2026Kafka producers in production
A Kafka producer appends records to topic partitions. Configuration and tuning with real numbers, idempotence and delivery guarantees, and the client-library decision that quietly matters most.
August 21, 2026Kafka Streams
Kafka Streams is a Java library for stream processing that runs inside your application, with state in local RocksDB stores and changelog topics. The operational realities, and where Flink wins.
August 21, 2026Kafka Streams documentation, mapped
The Kafka Streams documentation divides into four layers: API reference, configuration surface, state store internals, and the upgrade guide. Which layer answers which production question.
August 21, 2026Kafka topics
A Kafka topic is a named, append-only log, and in production it is the unit you operate on. The CLI verbs, partition and replication mechanics, retention, compaction and the troubleshooting moves.
August 21, 2026A Kafka topic example, fully specified
A production Kafka topic example: the exact creation command with durability properties, the same topic as Terraform and Strimzi code, the naming convention that scales, and the schema contract.
August 21, 2026Kafka topic vs partition
A topic is the logical name; a partition is the physical log. How replication, ordering, parallelism and key hashing follow the physical unit, and why keyed topics lock their count.
August 21, 2026A production Kafka tutorial
A Kafka tutorial for people who run it in production: zero-downtime upgrades, broker tuning, layered security, troubleshooting signals, and the client settings that decide delivery guarantees.
August 21, 2026Kafka use cases
The production Kafka use cases with the numbers behind them: real-time analytics, event-driven microservices, change data capture, event sourcing and log aggregation, plus the business case for each.
August 21, 2026Kafka KRaft
KRaft replaces ZooKeeper with a Raft quorum built into Kafka. The migration path and deadlines, controller sizing, the real scale limits, and day-2 operations for KRaft clusters.
August 21, 2026ksqlDB on Kafka
ksqlDB is the streaming SQL layer for Kafka: streams and tables in SQL, persistent queries with state in internal topics. The push-vs-pull trap, and where it stands as Confluent shifts to Flink.
August 21, 2026Kafka with Spring Boot
Spring Boot integrates with Kafka through spring-kafka: KafkaTemplate, @KafkaListener and auto-configuration. Production setup, dead letter topics, non-blocking retries, and listener tuning.
August 21, 2026Dead letter queues in Kafka: a consumer-side, Kafka-only approach
How to implement dead letter queues in Kafka: consumer-side, Kafka-only, scoped per consumer group, and why retry topics for transient failures are an anti-pattern.
August 19, 2026What is Apache Kafka?
Apache Kafka is an open-source distributed event streaming platform that stores records in ordered, partitioned, replayable logs. This is what it is, how the pieces fit, and when to choose it.
August 13, 2026
Comparisons Kafka UI: The Ultimate Guide
A Kafka UI is a web interface for managing Apache Kafka, giving operators visual control over topics, consumers, brokers, and connectors without the CLI.
June 27, 2026
Product Data lineage support in Factor Platform
Learn how Factor Platform brings OpenLineage metadata into your Kafka environment, making data ownership, PII classification, and lineage visible by default.
May 29, 2026
Industry What the IBM-Confluent deal means for Kafka users
IBM's $11B Confluent acquisition raises questions for Kafka users. Assess your lock-in risk across Schema Registry, managed connectors, and operational tooling.
March 26, 2026