Works on My Machine: How We Use AI to Reproduce Reported Bugs

Why It Matters

Automating bug reproduction accelerates fix cycles, cuts developer overhead, and improves SDK reliability for the broader developer community.

Key Takeaways

• Sentry built a Claude “repro” skill to auto‑generate SDK bug reproductions
• Skill parses GitHub issue, creates minimal project, runs test, and opens PR
• Automation cut hours of manual setup for complex Django/HTTPS proxy bugs
• Engineers keep human oversight to limit bot noise and ensure quality
• Skill leverages CLI tools and error‑handling for reliable execution

Pulse Analysis

Sentry supports a massive ecosystem of 159 SDKs across languages and frameworks, each generating its own stream of bug reports. Reproducing these issues often involves setting up language‑specific environments, scaffolding boilerplate applications, and configuring edge‑case scenarios such as HTTPS proxies or legacy dependencies. This manual triage is time‑consuming and can delay critical fixes, especially when engineers must juggle multiple repositories and maintain ad‑hoc test apps. The complexity of reproducing bugs has become a bottleneck for rapid iteration in open‑source SDK development.

Enter the Claude‑powered "repro" skill, a lightweight LLM agent that automates the most tedious parts of the reproduction workflow. By feeding a GitHub issue URL, the skill extracts language, framework, and SDK versions, then spins up a new repository branch, installs the appropriate tooling (uv, npm, bundle, etc.), and attempts to run a minimal example. If the attempt succeeds, it generates clear run‑through instructions and opens a pull request, optionally linking back to the original issue. In a recent Python/Django case, the skill produced a ready‑to‑run project that saved engineers several hours of configuration, allowing them to focus on the core fix rather than environment setup.

While full automation remains a future goal, Sentry is cautious about overwhelming developers with bot‑generated noise. The current approach balances AI‑driven efficiency with human oversight, ensuring that complex edge cases still receive a reviewer’s eye. This model showcases how LLMs can augment, rather than replace, engineering workflows—automating repetitive tasks while preserving quality control. As more organizations adopt similar agents, we can expect faster bug resolution cycles, reduced developer fatigue, and more reliable SDKs across the software supply chain.