“Backed by Science” Doesn’t Mean What You Think with Dr Stephen Cheung

Why It Matters

Understanding the limits of single‑study claims protects athletes from ineffective products and pushes the industry toward more rigorous, reproducible research, ultimately improving performance outcomes.

Key Takeaways

• Single study rarely proves product efficacy; replication essential.
• Small sample sizes in exercise science lead to underpowered results.
• Publication bias favors positive findings, suppressing null results.
• Statistical significance (p<0.05) can be a false positive artifact.
• Consumers should evaluate whole body of evidence, not marketing claims.

Summary

The episode tackles the overused claim “backed by science,” exposing how a lone study rarely validates a product’s performance. Host Trevor Connor and Dr. Stephen Cheung explain that true scientific confidence comes from multiple, independently replicated investigations, not isolated headlines.

Listeners learn that exercise‑science research often suffers from tiny participant pools, which reduces statistical power and inflates the chance of false‑positive findings. Publication bias further skews the literature, as journals preferentially publish studies showing an effect while null results remain on the shelf.

Cheung illustrates these points with concrete examples—heat‑adaptation protocols, core‑temperature wearables, and even a simple bicep‑curl trial—showing how a 5% alpha level can produce a “significant” result one in twenty times. He likens the interpretation of non‑significant outcomes to a criminal‑justice acquittal: absence of evidence isn’t evidence of absence.

The discussion warns athletes, coaches, and consumers to scrutinize the entire body of evidence before adopting new gear or training methods. Brands must present balanced data, and researchers need to prioritize replication and larger, well‑designed studies to restore credibility in sports‑performance science.

Original Description

When a product, wearable, supplement, or training method claims to be “backed by science,” what does that actually mean?

In this episode of Fast Talk, Dr. Stephen Cheung joins Trevor Connor and Chris Case to explain why a single published study is not the same as definitive proof—and why athletes and coaches need to look at the full body of evidence before trusting performance claims.

Dr. Cheung, editor-in-chief of the International Journal of Sports Physiology and Performance, breaks down the real-world limitations of sports science, including small sample sizes, publication bias, the replication crisis, statistical significance, underpowered studies, and the challenges of interpreting research in endurance performance. He also explains why studies showing “no effect” are often misunderstood or never published, creating a distorted picture of what actually works.

The conversation also explores how marketing can complicate scientific claims, especially around new endurance trends and technologies like heat training, core temperature sensors, wearables, nutrition products, and AI-generated research summaries. Dr. Cheung explains why multiple studies from multiple research groups matter, why meta-analyses can be useful but imperfect, and how coaches and athletes can become more discerning consumers of science.

You’ll also hear from Pat Warner of Stages Cycling on the challenges of balancing product development, independent testing, and marketing; Dr. Jamie Whitfield on the mental and physiological demands of heat training; and statistician Dr. Michael Rosenblatt on what makes a meta-analysis trustworthy.

In this episode, we cover:

Why “backed by science” can be misleading

How to evaluate sports science claims

Why one study is rarely enough

The replication crisis in exercise science

Publication bias and why “no effect” studies matter

The limits of wearable technology and performance devices

Heat training, hemoglobin mass, and endurance performance

Why meta-analyses can still be biased

Whether AI can accurately summarize scientific research

How athletes and coaches can decide what evidence to trust

For cyclists, triathletes, runners, coaches, and endurance athletes trying to make smarter decisions about training, recovery, nutrition, and technology, this episode is a guide to thinking more critically about the science behind performance.

Fast Talk Labs is your source for the science of endurance performance—cycling training, physiology, recovery, nutrition, and data-driven coaching tips to help athletes of all levels get faster.