Black Hat USA 2025 | More Flows, More Bugs: Empowering SAST with LLMs and Customized DFA

Why It Matters

By integrating LLMs with custom DFA enhancements, SAST becomes more automated and comprehensive, enabling faster detection of critical bugs that traditional static analysis often misses.

Key Takeaways

• LLM agents automate source and sink identification in frameworks.
• Customized DFA patches improve cross‑thread, reflection, and pass‑by‑value analysis.
• Expert rule filtering reduces false positives from LLM‑generated functions.
• Scanning 5,000 projects increased detected data flows by over 15%.
• New sink detection uncovered a critical SQL injection in Apache Traffic Control.

Summary

The Black Hat USA 2025 talk, presented by Yuan of Tencent Security Winding Lab, detailed a novel approach to strengthening static application security testing (SAST) by marrying large language models (LLMs) with a customized data‑flow analysis (DFA) engine. The session outlined the limitations of existing tools such as CodeQL—namely incomplete source‑sink coverage and language‑feature gaps—and introduced a three‑agent LLM workflow (discover, judge, validation) to automatically surface source and sink functions from open‑source frameworks.

Key insights included prompting LLMs with precise function characteristics, filtering results via confidence thresholds, and applying expert‑derived rules (public accessibility, non‑boolean returns, etc.) to prune false positives. On the DFA side, the team added “jump” steps and an additional‑value interface to bridge gaps in cross‑thread propagation, reflection handling, and Java’s pass‑by‑value semantics, enabling full source‑to‑sink path reconstruction where default analyses failed.

Concrete examples demonstrated the methodology: a missing flow in a Runnable demo was recovered by a jump from the constructor to the run method; reflection calls were resolved by tracking method instances and duplicating the data‑flow engine; and a pass‑by‑value scenario was fixed by mapping non‑post‑update and post‑update nodes. The research uncovered roughly 190 new source/sink definitions across 18 Go frameworks, and scanning over 5,000 open‑source projects yielded a 15% increase in detected data flows, including a previously hidden SQL‑injection vulnerability in Apache Traffic Control.

The implications are significant for DevSecOps pipelines: LLM‑augmented SAST dramatically reduces manual rule authoring, expands coverage of modern language features, and can be continuously validated against real‑world codebases. As static analysis tools adopt similar AI‑driven techniques, organizations can expect faster vulnerability discovery, lower false‑negative rates, and stronger overall security postures.

Original Description

Static Application Security Testing (SAST) plays a significant role in modern vulnerability discovery. For example, GitHub uses CodeQL to scan repositories. However, our analysis of over 100 real-world vulnerabilities has revealed that its detection performance is limited by two main factors: 1) incomplete source and sink coverage in built-in propagation rules, and 2) disruptions in data flow due to insufficient support for certain language features. In this talk, we will introduce a framework to empower SAST tools' capabilities to identify previously undetectable vulnerabilities and new CVEs.

First, we will demonstrate how to leverage Large Language Models (LLMs) to automatically identify sources and sinks from open-source frameworks. Second, we will introduce the implementation principles of CodeQL's Data Flow Analysis (DFA). By developing patches for the DFA's QL language library, we have addressed language feature challenges, including Java reflection handling, partial native method support, and value passing model optimization.

Our enhancements support 191 sources and sinks across 18 frameworks. Through comprehensive verification of over 5,000 repositories, we identified a more than 15% increase in data flows when utilizing existing rules, compared to results without the enhancements. Additionally, we reproduced over 50 historical CVEs that were undetectable by the original CodeQL due to a lack of language features support. Our research also uncovered 5 new CVEs (e.g., CVE-2024-45387) that the original CodeQL could not detect. We believe our work will greatly empower the detection capabilities of SAST tools.

By:

Yuan Luo | Senior Security Engineer, Tencent Security YunDing Lab

Zhaojun Chen | Senior Security Engineer, Tencent Security YunDing Lab

Yi Sun | Senior Security Engineer, Tencent Security YunDing Lab

Rhettxie Rhettxie | Senior Security Engineer, Tencent Security YunDing Lab

Presentation Materials Available at:

https://blackhat.com/us-25/briefings/schedule/index.html#more-flows-more-bugs-empowering-sast-with-llms-and-customized-dfa-45259