Half of AI Health Answers Are Wrong Even Though They Sound Convincing

Key Takeaways

• Five leading AI chatbots answered 250 health questions in a systematic test
• Overall, half of the responses were flagged as problematic or worse
• Grok performed poorest, with 58% problematic answers; ChatGPT 52%
• Vaccines and cancer queries yielded the most accurate AI-generated answers
• Nutrition and athletic performance topics saw the highest rate of AI errors

Pulse Analysis

The rise of conversational AI has transformed how patients seek medical information, offering instant, polished answers that often appear authoritative. Yet the allure of speed and accessibility masks a critical flaw: these systems lack the rigorous validation processes that underpin clinical guidance. As chatbots become embedded in consumer health workflows, the temptation to replace a physician’s counsel with a quick AI response grows, raising concerns about misinformation, misdiagnosis, and the erosion of trust in evidence‑based care.

The BMJ Open study subjected ChatGPT, Gemini, Grok, Meta AI and DeepSeek to a stress test of 250 real‑world health queries, spanning high‑stakes domains such as early‑stage cancer and vaccine safety. Independent experts rated each response, revealing that roughly one‑in‑five answers were highly problematic and half fell into a broader problematic category. While the bots performed comparably overall, Grok lagged with 58% problematic replies, and even the leading models like ChatGPT failed to produce reliable reference lists. Notably, the AI systems excelled on topics with dense, well‑structured literature—vaccines and cancer—yet faltered on nutrition and athletic performance, where evidence is fragmented and contradictory.

For stakeholders, the study signals an urgent call to action. Consumers must be educated to verify AI‑generated health advice against reputable sources, and clinicians should proactively discuss the limits of chatbot information during consultations. Regulators may need to establish standards for medical AI disclosures, ensuring that models flag uncertainty and provide verifiable citations. Meanwhile, developers are tasked with improving source attribution and integrating real‑time evidence checks, lest the technology’s promise be undermined by a steady stream of convincing yet inaccurate health recommendations.