AI

Generative AI in the Real World: Agentic Systems Fundamentals with Maarten Grootendorst

O’Reilly Media
O’Reilly Media•Jun 11, 2026

Why It Matters

Grasping the mechanics of agentic systems enables firms to build reliable, maintainable AI products while avoiding hidden costs. It also guides strategic choices about model openness and future‑proofing technology investments.

Key Takeaways

  • •Agents are LLMs looped with tools, memory, and guardrails.
  • •Embeddings and topic models remain valuable despite LLM dominance.
  • •Open-weight models offer transparency; proprietary models provide performance edge.
  • •Overreliance on black‑box code risks unmanageable technical debt.
  • •State‑space models could reshape attention mechanisms in future AI systems.

Pulse Analysis

Agentic systems are quickly becoming the backbone of enterprise AI, yet many leaders still view them as a black‑box extension of large language models. Grootendorst breaks down the concept: an LLM repeatedly invoked in a loop, equipped with external tools, a memory store, and safety guardrails. This architecture allows the model to act autonomously, retrieve up‑to‑date information, and adapt its behavior over time, bridging the gap between static text generation and dynamic decision‑making. Understanding these components demystifies deployment challenges and clarifies where human oversight remains essential.

Even as generative AI hype centers on chat‑based LLMs, foundational techniques such as embeddings and topic modeling retain practical relevance. Embeddings power similarity search, recommendation engines, and semantic clustering, while BERTopic‑style topic models provide interpretable insights from massive corpora. The debate between open‑weight and proprietary models adds another layer: open models grant transparency and customization, but proprietary offerings often deliver superior performance and integrated tooling. Emerging state‑space models promise more efficient attention mechanisms, potentially reducing compute costs and enabling real‑time agentic applications. Companies must weigh these trade‑offs against their data privacy, latency, and scalability requirements.

Grootendorst warns that a generation of developers may be shipping code they cannot fully read, creating a hidden mountain of technical debt. Without a solid intuition for how LLMs process prompts, manage memory, and interact with tools, teams risk brittle pipelines that fail under edge cases. Investing in education, open‑source experimentation, and robust monitoring can mitigate these risks. As the industry moves toward more autonomous AI agents, a balanced approach—leveraging both cutting‑edge models and time‑tested techniques—will be key to sustainable innovation.

Original Description

BERTopic creator and Google DeepMind developer relations engineer Maarten Grootendorst has spent years helping practitioners build intuition for how AI systems actually work—not just how to prompt them. Maarten joined Ben Lorica to cover the enduring relevance of embeddings and topic models in an LLM-dominated world, his hot take that agents are essentially just an “LLM in a for loop with some tools, some memory, and perhaps some guardrails," and what separates genuine agentic behavior from a well-constructed pipeline. They also get into the practical trade-offs between open weight and proprietary models, the future of state space models and attention, and why Maarten worries that a generation of builders shipping code they can't read may be storing up technical debt they can't repay. "If you don't really know how an LLM works," he says, "that intuition [about how to use it effectively] is much more difficult to develop."
Follow O'Reilly on:

Comments

Want to join the conversation?

Loading comments...