EP264 Measuring Your (Agentic) SOC: Two Security Leaders Walk Into a Podcast

In this episode the hosts and guests dissect why classic SOC metrics such as mean time to detect (MTTD) and mean time to respond (MTTR) are losing relevance. They argue that these time‑only indicators reward faster button clicks rather than effective incident resolution, and they highlight the need for quality‑focused measurements that capture automation success, false‑positive rates, and ticket accuracy. By reframing metrics around outcomes instead of raw speed, security teams can avoid the "teaching to the test" pitfall that many legacy programs fall into.

The conversation then shifts to the transformative role of AI and agentic automation in modern security operations. Guests describe a layered "triangle" or "pyramid" model that maps maturity across infrastructure, applications, and data, with each layer receiving its own detection and response metrics. As automation pushes response times from hours to seconds, the goalposts move: new metrics now blend time, quality, and AI‑driven confidence scores. Real‑world examples include automated ticket de‑duplication, AI‑evaluated ticket quality, and dynamic service‑level objectives that adapt as agents become more capable.

Finally, the panel tackles regulatory and audit challenges that accompany non‑deterministic AI decisions. Regulators expect auditable evidence of how alerts are handled, even when an AI model classifies an event as low‑risk. Organizations therefore maintain human‑in‑the‑loop safeguards, track the proportion of incidents resolved without human interaction, and measure the cost differential between human and machine effort. By publishing transparent metrics—such as automation accuracy, incident‑handling percentages, and compliance with standards like DORA—companies can demonstrate both operational efficiency and regulatory readiness while continuing to refine their SOC maturity roadmap.