Your LLM Issues Are Really Data Issues

The episode opens with Harsha Chintalapani explaining why large language models falter when faced with real‑time, structured production data. He cites Uber’s infamous "trips" table mishap, where analysts accessed stale or duplicate tables, and the chaos that follows schema changes, ambiguous definitions of core concepts like "location," and GDPR‑driven manual data classification. These examples illustrate that the bottleneck isn’t model capability but the underlying data ecosystem—its semantics, freshness, and discoverability—especially when organizations attempt to feed raw enterprise data directly into LLMs.

To address these pain points, Harsha describes the OpenMetadata platform, a schema‑first knowledge graph that automatically ingests metadata from sources such as Hive, Snowflake, Kafka, and Airflow. By standardizing table definitions, ownership, data‑quality signals, and lineage relationships, the system turns opaque data lakes into searchable catalogs. Analysts can instantly locate the correct customer table, verify freshness, and understand privacy classifications without manual hunting. This metadata‑centric approach not only accelerates business‑intelligence workflows but also prepares data for AI consumption, ensuring that models receive well‑documented, trustworthy inputs.

Finally, the conversation broadens to emphasize that these challenges transcend Uber’s scale. Start‑ups and mid‑size firms face the same ownership ambiguity and semantic drift once they democratize data access. The key takeaway for leaders is to embed explicit, human‑crafted semantics into metadata repositories before deploying AI agents. By doing so, organizations can reduce onboarding time, improve decision‑making speed, and unlock the true potential of LLMs in production environments. The future of data governance lies in automated metadata capture paired with clear, organization‑wide definitions.