Small Models Also Found the Vulnerabilities that Mythos Found

Why It Matters

The findings show organizations can achieve high‑impact vulnerability detection without costly frontier models, shifting investment toward robust orchestration and maintainer trust rather than exclusive AI APIs.

Key Takeaways

• Small open‑weight models detect Mythos’s FreeBSD exploit
• Cybersecurity capability scales unevenly across model sizes
• System scaffolding, not model, provides the real moat
• Exploit construction remains a gap for cheaper models
• Maintainer trust, not raw AI power, drives adoption

Pulse Analysis

Anthropic’s Mythos preview generated buzz by announcing an AI model that autonomously uncovered and crafted exploits for thousands of zero‑day vulnerabilities, backed by a $100 million credit pledge and a $4 million donation to open‑source security initiatives. The public narrative suggested that a single, restricted‑access model could revolutionize vulnerability discovery, positioning Mythos as a watershed moment for AI‑driven cybersecurity. While the technical blog highlighted sophisticated exploit chains across operating systems and browsers, the broader industry impact hinges on whether such capabilities are truly exclusive or can be replicated with more accessible tools.

In a systematic evaluation, AISLE’s chief scientist Stan Fort isolated the code snippets used in Mythos’s showcase and ran them through a spectrum of inexpensive, open‑weight models. Remarkably, a 3.6 billion‑parameter model costing just $0.11 per million tokens correctly identified the FreeBSD NFS buffer overflow, and a 5.1 billion‑parameter model reconstructed the complex OpenBSD SACK vulnerability chain. Across three targeted tests—false‑positive discrimination, FreeBSD exploit detection, and OpenBSD bug analysis—the performance rankings reshuffled, demonstrating that model size does not linearly predict security reasoning ability. This jagged capability landscape underscores that the decisive factor is the surrounding pipeline: code‑scoping, triage, validation, and integration with maintainer workflows.

For enterprises and open‑source projects, the takeaway is clear: investing in a robust orchestration system and cultivating deep security expertise yields greater returns than locking into a single, expensive AI service. Cheap, high‑throughput models can blanket large codebases, while expert‑crafted scaffolds filter, verify, and prioritize findings, delivering the trust needed for maintainer acceptance. The remaining frontier—automated exploit construction—still favors specialized models like Mythos, but for most defensive use cases, discovery and patch generation are already within reach using affordable AI. Organizations that build these systemic capabilities now will secure a competitive edge as the AI security ecosystem matures.