What ClickHouse Learned From a Year of Coding with AI Agents

Why It Matters

AI agents now deliver measurable productivity gains for large‑scale software teams, turning millions of daily test runs into manageable workloads and freeing senior engineers for higher‑impact work, which can be a decisive competitive edge in talent‑tight markets.

Key Takeaways

• Claude Opus 4.5 enabled daily AI‑assisted work on large C++ codebase
• Agents cut flaky test investigations from 200 to 3‑5 daily findings
• Automated code reviews catch leaks, race conditions, freeing humans for architecture
• Start with boilerplate and merge conflicts before tackling complex tasks
• Maintain at least two model providers to avoid downtime

Pulse Analysis

AI‑assisted coding has moved from ad‑hoc copy‑paste snippets to integrated agents that can read a repository, execute commands, and commit changes. The New Stack article outlines three maturity levels that mirror the broader industry: simple chat‑based copy‑pasting, CLI/IDE‑embedded agents, and fully autonomous multi‑agent loops. ClickHouse’s experience illustrates how the release of Anthropic’s Claude Opus 4.5 in late 2025 shifted the technology from experimental to production‑ready for a massive C++ codebase, prompting the team to embed agents in daily workflows.

Since the upgrade, agents have become a productivity multiplier. In the CI pipeline that runs 20‑80 million tests per day, the team reduced daily flaky‑test investigations from roughly 200 findings to just three‑to‑five per ten million runs, thanks to autonomous PRs that diagnose and fix flakiness. Automated code‑review bots now surface resource leaks, race conditions, and edge‑case bugs, allowing senior engineers to focus on architecture rather than line‑level defects. Merge‑conflict resolution is now handled almost entirely by agents, delivering near‑perfect outcomes and freeing developers for higher‑value work.

The article’s seven‑point playbook offers a roadmap for firms eager to replicate ClickHouse’s gains. Starting with low‑risk tasks such as boilerplate updates and conflict resolution builds trust before tackling complex bug fixes. Maintaining multiple model providers mitigates service outages, while rigorous validation through expanded testing safeguards against confidently wrong suggestions. As autonomous multi‑agent loops mature, early adopters who embed agents into CI and triage pipelines will likely widen the productivity gap with slower teams. In a market where engineering talent is scarce, AI‑driven coding assistants are becoming a strategic differentiator.