How Easily Can Russian Propaganda Fool AI Models? A New Benchmark Finds Out

Why It Matters

The findings expose stark differences in how leading AI models handle state‑sponsored disinformation, influencing trust, regulatory scrutiny, and competitive positioning in a market where misinformation risk is a key differentiator.

Key Takeaways

• Anthropic Claude models topped the benchmark with scores above 95
• Nvidia Nemotron 3 and Alibaba Qwen 3.6 Plus placed second
• Mistral models fell in the bottom third, reflecting high misinformation rates
• Russian disinformation networks feed AI systems millions of propaganda articles
• OpenAI shut down a Russian ChatGPT campaign before Germany’s election

Pulse Analysis

The Estonian Language Institute’s new benchmark shines a light on a growing security concern: whether AI chatbots can be weaponized by foreign propaganda machines. By posing 75 neutral, biased and manipulative questions in English, Russian and Ukrainian, the study isolates the model’s internal reasoning, stripping away external tools that could otherwise correct errors. Claude Opus 4.5, calibrated by disinformation experts at Propastop, acted as the judge, scoring each response on a five‑point scale where a score of one indicates outright replication of Russian talking points. This methodology offers a rare, controlled glimpse into the raw susceptibility of large language models.

Anthropic’s Claude family emerged as the clear leaders, with Claude Fable 5 achieving a 95.2 overall score, followed closely by Claude Opus 4.7. Nvidia’s Nemotron 3 and Alibaba’s Qwen 3.6 Plus secured the next spots, while Mistral’s latest Medium 3.5 languished in the bottom third. The results echo a Newsguard analysis that flagged Mistral’s misinformation rate at 36.7 percent, a troubling figure as the French firm negotiates a €3 billion (≈$3.27 billion) funding round at a €20 billion (≈$21.8 billion) valuation. For investors and enterprise buyers, model resilience to disinformation is becoming a decisive factor in procurement decisions.

The broader implication is stark: state‑backed actors like Russia’s Pravda network are deliberately flooding AI systems with millions of propaganda articles to corrupt training data. OpenAI’s recent shutdown of a Russian campaign that used ChatGPT to sway Germany’s federal election underscores the real‑world stakes. As regulators tighten oversight on AI‑generated content, providers that can demonstrably filter out hostile narratives will gain a competitive edge, while those lagging risk reputational damage, legal exposure, and loss of market share.