a16z Podcast
Columbia CS Professor: Why LLMs Can’t Discover New Science
a16z Podcast
•October 13, 2025•50 min
0
a16z Podcast•Oct 13, 2025
Key Takeaways
- •LLMs reduce complex knowledge to low‑entropy token manifolds.
- •RAG originated from a cricket stats interface hack.
- •Precise prompts lower prediction entropy, boosting answer confidence.
- •Chain‑of‑thought splits tasks into low‑entropy steps.
- •LLM capabilities plateau; new paradigms needed for breakthroughs.
Pulse Analysis
In this episode, a Columbia computer‑science professor argues that true artificial general intelligence will only emerge when machines can create new scientific theories, not just remix existing data. The conversation pivots to Vishal Misra’s accidental invention of retrieval‑augmented generation (RAG) while fixing a cricket‑stats web form. By pairing natural‑language queries with a curated database of example prompts, Misra demonstrated that large language models could reliably produce correct SQL statements despite limited context windows, foreshadowing the RAG wave that now powers many enterprise AI products.
The hosts dive deep into Misra’s formal models, beginning with the "matrix abstraction" that treats every prompt as a row in a massive, sparse probability matrix over the model’s vocabulary. This view reveals how LLMs compress a high‑dimensional stochastic world into Bayesian manifolds, where prediction entropy governs confidence. High‑information prompts shrink the manifold, yielding low‑entropy token distributions and more reliable outputs. The discussion also explains why chain‑of‑thought prompting works: it breaks a problem into a sequence of low‑entropy steps, effectively guiding the model along a well‑defined path within the manifold.
Finally, the panel reflects on the apparent plateau in LLM capabilities. While models have become faster, larger, and more polished, they have not crossed a fundamental performance threshold. To achieve genuine AGI—machines that can discover new physics or mathematics—researchers must develop new paradigms beyond current transformer‑based architectures. For business leaders, this means investing in hybrid systems that combine retrieval, formal reasoning, and domain‑specific algorithms, rather than relying solely on ever‑larger language models.
Episode Description
From GPT-1 to GPT-5, LLMs have made tremendous progress in modeling human language. But can they go beyond that to make new discoveries and move the needle on scientific progress?
We sat down with distinguished Columbia CS professor Vishal Misra to discuss this, plus why chain-of-thought reasoning works so well, what real AGI would look like, and what actually causes hallucinations.
Resources:
Follow Dr. Misra on X: https://x.com/vishalmisra
Follow Martin on X: https://x.com/martin_casado
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://x.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Podcast on Spotify
Listen to the a16z Podcast on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Show Notes
0
Comments
Want to join the conversation?
Loading comments...