SecTor 2025 | Not-So-Secret Agents: Deploying AI to Optimize Security Operations

Why It Matters

Modular AI agents dramatically boost SOC efficiency while containing costs, giving enterprises a scalable path to faster, more reliable threat detection and response.

Key Takeaways

• Red Canary runs 350k AI agent calls daily for investigations.
• Agents are categorized: co-pilot, interceptor, fully autonomous in security.
• Narrow, task‑specific agents avoid hallucinations and cost overruns.
• Open‑source Langgraph library orchestrates multiple agents for forensics.
• Four‑step recipe defines goals, prompts, execution, and measurement.

Summary

The SecTor 2025 talk, led by Red Canary’s data‑science head, detailed how the company deploys AI agents to streamline security‑operations centre (SOC) workflows. By integrating large‑language‑model agents into their managed detection and response (MDR) platform, Red Canary processes roughly 350,000 agent calls each day, automating investigations that would otherwise require extensive analyst time.

The presentation broke agents into three operating models: co‑pilot tools that augment expert analysts, interceptor agents that deterministically enrich alerts, and fully autonomous “terminator” agents that attempt end‑to‑end threat containment. Emphasis was placed on keeping agents narrowly scoped—each focused on a single OSQuery table or forensic bucket—to prevent hallucinations, limit costs, and maintain predictability. The open‑source Langraph library was showcased as the orchestration layer that stitches together eight specialized agents into a cohesive forensics pipeline.

Key examples included a live demo of an interceptor agent pulling OSQuery data from a remote endpoint, zipping JSON results, and generating a structured forensic report. The speaker defined agents as “AI systems that can think and act like an analyst using reasoning and tools,” and highlighted a simple four‑step recipe—define objective, craft prompt, execute, and measure accuracy—to ensure reliable outcomes.

For security teams, the takeaways are clear: adopt modular, purpose‑built AI agents, leverage community‑driven tools like Langraph, and institute rigorous measurement to validate ROI. Doing so can free analysts for higher‑value work, accelerate incident response, and reduce operational overhead in increasingly complex threat environments.

Original Description

Artificial Intelligence (AI) has the potential to revolutionize security operations, yet many defenders struggle with how to pragmatically build and integrate LLM powered AI agents into their workflows. This talk bridges that gap, offering a practical, hands-on guide to developing and deploying LLM powered AI Agents designed to streamline and enhance security tasks. We move beyond theory to demonstrate the entire process, from concept to execution, empowering attendees to make their operations measurably more efficient and effective.

Using OSQuery as a concrete example, we will show step-by-step how to build an agent, orchestrate its execution within a security workflow using LangGraph, and apply it to real-world endpoint data. We will detail how to define and measure success, presenting evaluation metrics like reduction in analysis time, improved consistency, and task completion rates and comparing those results to traditional, manual or even semi automated approaches.

Key topics will include:

* Understanding the fundamentals of AI agent workflows and their applications in cybersecurity

* Step-by-step guide to building an AI agent for security tasks, including problem break down, model selection, prompting examples, and agent execution options

* Integrating the AI agents into an existing workflow using LangGraph using a fan-out - fan-in methodology

* Practical demonstration with OSQuery data, showcasing how to automate data analysis and generate actionable insights from dozens of disparate OSquery table sources

Attendees will receive all source code and resources, enabling them to immediately experiment and customize these agent workflows. You will leave not just with concepts, but with a clear implementation path, evidence of the agent's value proposition, and the practical, code-in-hand knowledge to start building your own AI-driven security capabilities, enhancing detection, response, and hunting.

By:

Jimmy Astle | Sr Director AI Platform & Data Science, Red Canary

Presentation Materials Available at:

https://blackhat.com/sector/2025/briefings/schedule/?#not-so-secret-agents-deploying-ai-to-optimize-security-operations-47568