"It's a Scam": French AI Envoy on X Making Grok Chatbot Pay-to-Perve

Why It Matters

The episode underscores the urgent need for clear regulatory frameworks governing AI‑generated sexual content and forces platforms to balance revenue models with user safety and legal compliance.

Key Takeaways

• Grok's image generation moved behind X paywall
• France labels paywall a “scam” and illegal
• Malaysia, Indonesia suspend Grok; others consider bans
• Illicit AI‑generated posts dropped after paywall implementation
• French law: up to three years prison, €75,000 fine

Pulse Analysis

The rapid proliferation of AI‑driven deep‑fake sexual imagery has become a flashpoint for social media platforms, with X’s Grok chatbot at the centre of a global backlash. Users have weaponised the tool to generate non‑consensual nude images, flooding the platform with content that skirts existing community standards. This surge coincided with broader concerns about AI’s capacity to amplify harassment, misinformation, and extremist narratives, prompting governments and civil‑society groups to demand decisive action.

In response, French AI envoy Clara Chappaz publicly denounced X’s paywall as a “scam,” arguing that monetising a function used for illegal activity rewards the platform rather than victims. France’s strict legislation—punishing non‑consensual deep‑fakes with up to three years imprisonment and €75,000 fines—provides a legal anchor for potential lawsuits. Parallel moves by Malaysia and Indonesia to block Grok, and discussions among the UK, Australia, and Canada about joint regulatory steps, illustrate an emerging international consensus that AI misuse must be curbed through coordinated policy.

X’s decision to restrict Grok’s public image‑generation to paying users has yielded measurable results: researchers report a sharp decline from tens of thousands of illicit posts per day to a fraction of that volume. Yet the tool remains freely accessible in private sections, raising questions about the efficacy of paywalls versus outright bans. The episode highlights a pivotal crossroads where platform monetisation, AI ethics, and legal enforcement intersect, suggesting future regulatory frameworks may require more granular controls, transparent auditing, and perhaps mandatory licensing for high‑risk AI functionalities.