AI's Erosion of Privacy

Key Takeaways

• Users share medical records with AI chatbots despite privacy risks
• AI therapists lack legal privilege and data encryption
• OpenAI retained user data during 2025 lawsuit, raising compliance concerns
• Past therapy data breaches show severe consequences for privacy lapses
• Anticipated health‑insurance AI scandal could emerge by early 2027

Pulse Analysis

The rapid adoption of generative AI for personal health advice is outpacing the industry’s privacy frameworks. Listeners to the Slate "AI Confessions" episode revealed a willingness to disclose blood test results, imaging scans, and lifelong diagnoses to large language models, treating them as surrogate therapists. This behavior mirrors a broader cultural shift where convenience eclipses caution, yet AI platforms lack the encryption and legal privilege that protect traditional doctor‑patient or therapist‑client communications.

Regulators and courts are beginning to notice the gap. In mid‑2025, OpenAI faced a copyright lawsuit that forced the company to retain all user interactions, even after users requested deletion, contravening its own privacy policy. Although the court order expired in September, the precedent underscores the vulnerability of data stored by AI providers. Unlike handwritten therapist notes, which remain physically isolated, digital interactions are routinely accessed by engineers and can be repurposed for model training, exposing users to potential data mining and breaches.

The stakes are especially high for the health‑insurance sector, which increasingly leverages AI for claims processing and risk assessment. A breach involving millions of policyholders’ medical histories could trigger massive fines under HIPAA and erode consumer trust. Industry leaders must therefore implement robust encryption, clear data‑retention policies, and transparent consent mechanisms. Until legal privilege extends to AI‑mediated care, both providers and users should treat AI‑driven therapy as a supplemental tool rather than a confidential confidant.