Continuous Delivery (Dave Farley)

Is "Testing in Production" Actually the Safest Way to Ship?

The podcast episode explores whether testing in production is the safest way to ship software, contrasting disposable code generated by AI with durable, production‑tested code that underpins critical systems. Speakers argue that incremental, observable changes are essential; observability‑driven development, continuous deployment, and separating deploy from release create fast feedback loops that let engineers verify behavior in live environments. They also critique the myth of hiring only top‑percent engineers, citing Google’s Project Aristotle and Belbin’s team‑role research that show trust, diversity, and shared responsibility outperform raw talent. A memorable line from the conversation: “There’s only one way to test – in prod or live a lie,” underscoring the necessity of real‑world validation. The hosts cite Coinbase’s brag about hiring the 0.1% of applicants and contrast it with the need for sociotechnical systems that give every engineer rapid, safe feedback. The takeaway for leaders is to invest in observability tooling, automate safe rollbacks, and cultivate cultures where failure is a learning signal. By doing so, organizations can ship more frequently, accelerate skill development, and reduce reliance on elite hiring as a shortcut to performance.