This AI Agent Debugs Your Entire Data Pipeline 🤯

Andreas Kretz (Learn Data Engineering)
Andreas Kretz (Learn Data Engineering)•May 27, 2026

Why It Matters

This approach unifies structured and unstructured observability, letting teams quickly surface root causes and trends across millions of log lines using natural-language queries—reducing time-to-diagnosis and improving pipeline reliability. It demonstrates practical integration of vector search into traditional databases for actionable RAG-powered analytics.

Summary

A demo shows how to build an AI agent that debugs data pipelines by combining relational SQL queries and vector-based RAG search inside an Oracle 26 AI database. The presenter spins up an Oracle AI container via Docker Compose, loads pipeline job and log rows (including embeddings for log messages), and connects a local LLM (Ollama) via LangChain. The agent uses three tools—SQL search, vector log search, and a hybrid SQL-then-vector flow—to answer queries like failure counts and to generate a consolidated diagnostic report. The notebook and full 20-minute walkthrough are available on the presenter’s YouTube channel and Oracle AI Developer Hub.

Original Description

Built an AI agent that debugs your data pipelines. Relational + RAG in one db!
Full tutorial on my channel 👆

Comments

Want to join the conversation?

Loading comments...