Apache Iceberg is an open table format built to bring reliable, SQL-like tables to huge analytic datasets stored in an object store like S3, with ACID transactions, schema evolution, hidden partitioning, and time travel built in.
This hub is the entry point to what we have written about running Iceberg in production. It is new and will grow section by section as we publish more, starting with Iceberg use cases: real deployments, with the architecture and the numbers behind them.
Everything the Iceberg cluster covers
Companies running Apache Iceberg in production and the architectures behind their deployments, indexed as we publish them.
Learn moreHow Netflix moved its S3 data warehouse off Hive onto an Iceberg-only architecture, and how Maestro, Psyberg, and its Cassandra data movement engine lean on Iceberg's own snapshot and partition metadata.
Learn moreHow Apple built GDPR- and DMA-compliant row-level deletes on Iceberg tables holding tens of petabytes of data, sourced from a peer-reviewed paper by eight Apple engineers and named conference talks.
Learn moreHow Airbnb moved its Kafka-fed warehouse ingestion pipeline off Tez and Hive onto Spark 3 and Apache Iceberg, consolidating tables via partition-spec evolution and cutting compute by over 50%.
Learn moreHow Pinterest rebuilt CDC ingestion and ML feature backfills on Apache Iceberg, cutting pipeline latency from 24+ hours to minutes and backfill time from 140 days to 26.
Learn moreHow LinkedIn's OpenHouse control plane governs 300,000+ Apache Iceberg tables holding over an exabyte of data, with automated compaction and cross-region replication.
Learn moreSeven Kafka and Flink tools scored on operating the Kafka to Iceberg pipeline, from Connect sink tasks and sink lag to Flink checkpoints, access and audit.
Learn moreWhy a Kafka to Iceberg pipeline writes many tiny data files, how to measure it, and how to fix it with compaction and sink settings.
Learn moreWhy a streaming-fed Iceberg table keeps growing in storage and metadata, how to measure it with the metadata tables, and how to expire snapshots and remove orphan files without breaking time travel or the writer.
Learn moreWhy a streaming writer fails to commit to an Iceberg table with CommitFailedException or exhausted retries, how optimistic concurrency works, how to find the conflicting job, and how to fix it.
Learn moreWhy a Kafka Connect or Flink writer fails or drops data when a producer adds, renames, retypes or removes a field, what Iceberg schema evolution allows, how registry compatibility interacts, and how to fix and backfill.
Learn moreWhy a reader finds no data or fails with table not found while the Kafka to Iceberg writer commits, how to compare catalogs, warehouse paths and metadata locations, and how to fix it without registering one table in two catalogs.
Learn more