The Analytics Engineering Podcast

The Iceberg Ecosystem Today (W/ Anders Swanson)

In this episode, Anders Swanson, a developer experience advocate at dbt Labs, walks through the current state of the Apache Iceberg ecosystem, covering how open‑source and cloud vendors are converging on shared standards, the rise of external catalog integrations, and the performance challenges of metadata services. He explains key concepts such as query engines, object stores, and the distinction between internal and external catalogs, highlighting recent innovations like Snowflake’s catalog‑link databases and Databricks’ lake federation. Swanson also discusses practical trade‑offs—mirroring versus on‑the‑fly metadata fetching—and why some platforms (e.g., DuckDB) choose strict limits to maintain reliability. The conversation offers a realistic view of what data teams can expect when building production workloads on Iceberg today.