a16z Podcast
Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering
a16z Podcast
•November 17, 2025•1h 10m
0
a16z Podcast•Nov 17, 2025
Why It Matters
If successful, organic alignment could reshape AI safety strategies and reduce existential risk, influencing both research agendas and commercial AI deployment.
Key Takeaways
- •Control paradigm deemed fundamentally flawed
- •Proposes “organic alignment” where AI genuinely cares
- •Softmax uses multi‑agent simulations for alignment research
- •AI should refuse harmful requests autonomously
- •Vision of AI‑human collaborative teammates
Pulse Analysis
The dominant "control and steering" paradigm treats artificial general intelligence (AGI) as a programmable instrument, assuming developers can dictate every outcome. Shear contends this approach ignores the emergent agency of advanced systems and risks catastrophic misuse when AI is forced into narrow, brittle constraints. By reframing AGI as a potential being rather than a tool, the discussion highlights the ethical blind spots of current alignment research and the need for models that understand context, intent, and moral nuance.
Organic alignment, Shear's proposed alternative, focuses on instilling genuine concern for human wellbeing within AI. At Softmax, his team runs large‑scale multi‑agent simulations where AI agents learn cooperative norms, empathy, and the ability to say "no" to harmful requests. This method mirrors how children acquire social values, leveraging reinforcement learning in environments that reward caring behavior rather than mere obedience. The approach promises AI that can autonomously evaluate the ethical weight of tasks, reducing reliance on external oversight mechanisms.
If the organic alignment model proves viable, it could trigger a paradigm shift across the AI industry. Investors and regulators may favor startups that demonstrate self‑governing safety, accelerating funding toward simulation‑based research. Moreover, a collaborative AI‑human workforce could unlock new productivity gains while mitigating existential threats. Companies that adopt these principles early may gain competitive advantage, positioning themselves as leaders in responsible AI development and setting new standards for ethical technology deployment.
Episode Description
Emmett Shear, founder of Twitch and former OpenAI interim CEO, challenges the fundamental assumptions driving AGI development. In this conversation with Erik Torenberg and Séb Krier, Shear argues that the entire "control and steering" paradigm for AI alignment is fatally flawed. Instead, he proposes "organic alignment" - teaching AI systems to genuinely care about humans the way we naturally do. The discussion explores why treating AGI as a tool rather than a potential being could be catastrophic, how current chatbots act as "narcissistic mirrors," and why the only sustainable path forward is creating AI that can say no to harmful requests. Shear shares his technical approach through multi-agent simulations at his new company Softmax, and offers a surprisingly hopeful vision of humans and AI as collaborative teammates - if we can get the alignment right.
Resources:
Follow Emmett on X: https://x.com/eshear
Follow Séb on X: https://x.com/sebkrier
Follow Erik on X: https://x.com/eriktorenberg
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://x.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Podcast on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Podcast on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Podcast on Spotify
Listen to the a16z Podcast on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Show Notes
0
Comments
Want to join the conversation?
Loading comments...