a16z Podcast
What Comes After ChatGPT? The Mother of ImageNet Predicts The Future
a16z Podcast
•December 5, 2025•1h 1m
0
a16z Podcast•Dec 5, 2025
Key Takeaways
- •Compute scaling enables next‑generation 3D world models.
- •Spatial intelligence requires generative models beyond language.
- •Open datasets like Behavior drive collaborative AI research.
- •Academia lacks resources; needs novel hardware‑friendly architectures.
- •Marvel model creates explorable 3D worlds from textures.
Pulse Analysis
The episode opens by framing deep learning’s evolution as a story of ever‑greater compute. Fei‑Fei Li and Justin Johnson argue that the massive GPU clusters now available—orders of magnitude beyond AlexNet’s era—make it feasible to train spatial intelligence models that generate full 3D environments. Their new Marvel system demonstrates this shift, turning texture images into explorable worlds for gaming, VFX, and film, and signals a broader move from pure language models toward visual and spatial reasoning.
A second theme centers on the balance between open science and commercial pressure. Li highlights Stanford’s Behavior benchmark, a public dataset for robotic learning, as a modern analogue to ImageNet’s open challenge model. While industry startups push rapid productization, the speakers stress that a healthy ecosystem needs both proprietary advances and shared resources. Funding structures, policy initiatives like the National AI Resource (NAYER) bill, and cross‑sector collaborations are presented as essential to sustain innovation without stifling academic freedom.
Finally, the conversation turns to future hardware and research culture. Johnson warns that today’s GPU‑centric matrix‑multiplication paradigm may hit scaling limits, urging exploration of new primitives suited to distributed clusters. Li echoes the need for “wacky” academic ideas—novel architectures, algorithmic breakthroughs, and interdisciplinary work—that can thrive despite limited compute budgets. Together, they envision a next decade where academia, bolstered by public compute clouds and open datasets, fuels the breakthroughs required for truly immersive, physics‑aware world models.
Episode Description
Fei-Fei Li is a Stanford professor, co-director of Stanford Institute for Human-Centered Artificial Intelligence, and co-founder of World Labs. She created ImageNet, the dataset that sparked the deep learning revolution.
Justin Johnson is her former PhD student, ex-professor at Michigan, ex-Meta researcher, and now co-founder of World Labs.
Together, they just launched Marble—the first model that generates explorable 3D worlds from text or images.
In this episode Fei-Fei and Justin explore why spatial intelligence is fundamentally different from language, what's missing from current world models (hint: physics), and the architectural insight that transformers are actually set models, not sequence models.
Resources:
Follow Fei-Fei on X: https://x.com/drfeifei
Follow Justin on X: https://x.com/jcjohnss
Follow Shawn on X: https://x.com/swyx
Follow Alessio on X: https://x.com/fanahova
Stay Updated:
If you enjoyed this episode, please be sure to like, subscribe, and share with your friends.
Follow a16z on X: https://x.com/a16z
Follow a16z on LinkedIn:https://www.linkedin.com/company/a16z
Follow the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Follow the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details, please see http://a16z.com/disclosures.
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Podcast on Spotify
Listen to the a16z Podcast on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Show Notes
0
Comments
Want to join the conversation?
Loading comments...