Bubble or No Bubble, AI Keeps Progressing (Ft. Relentless Learning + Introspection)

Why It Matters

If sustained, these advances would let deployed LLMs personalize and improve from real-world use without wholesale retraining, reshaping product roadmaps, competitive dynamics, and trust/governance needs across AI services.

Summary

The video argues against the view that AI progress has plateaued, highlighting recent research that points to practical paths for continual and nested learning in language models. It summarizes a Google paper proposing a 'hope' architecture that flags novel prediction errors and stores enduring signals in updatable memory layers, enabling models to learn from ongoing user interactions while protecting core knowledge. The presenter notes limits — these techniques don’t by themselves fix hallucinations, scale to trillion-parameter models is unproven, and gating learned content remains a challenge — and suggests reinforcement learning and safety gating as complementary steps. He also flags Anthropic’s work on model introspection as evidence that understanding and steering internal concepts is increasingly tractable.

Original Description

Don’t let headlines about bubbles distract you from the real avenues of progress being explored in AI every week, including what had been thought to be a long-term blocker - continual learning (learning on the fly).

https://app.grayswan.ai/ai-explained

This, plus models introspecting (hesitate before you berate), Nano Banana 2 possibly spotted, Chinese imagen and more.

AI Insiders ($9!): https://www.patreon.com/AIExplained

Chapters:

00:00 - Introduction

01:26 - Continual Learning (Nested Learning / HOPE)

07:00 - Introspection

10:54 - Image-Gen Progress

Nested Learning Post: https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/

Nested Learning Paper: https://abehrouz.github.io/files/NL.pdf

Original Titans Paper: https://arxiv.org/pdf/2501.00663

Siri News: https://www.bloomberg.com/news/articles/2025-11-05/apple-plans-to-use-1-2-trillion-parameter-google-gemini-model-to-power-new-siri

Introspection: https://www.anthropic.com/research/introspection

Full Paper: https://transformer-circuits.pub/2025/introspection/index.html#mechanisms

Earlier Work: https://www.anthropic.com/research/mapping-mind-language-model

https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html

Release Post: https://x.com/AnthropicAI/status/1983584136972677319

https://lmcouncil.ai

Non-hype Newsletter: https://signaltonoise.beehiiv.com/

Podcast: https://aiexplainedopodcast.buzzsprout.com/