GDS Reports Achieving 90% Accuracy in GOV.UK Chat Pilots

Why It Matters

The high accuracy demonstrates that government‑grade AI can meet rigorous public‑service standards, potentially transforming citizen access to information while setting a benchmark for safe AI deployment in the public sector.

Key Takeaways

• 90% accuracy across all topics, up from 76%
• 10,000+ users asked 26,000 questions in pilots
• Average response time 10.7 seconds, speed‑accuracy trade‑off
• 73% users found assistant useful; 64% satisfied
• 508 jailbreak attempts blocked by safety guardrails

Pulse Analysis

Government digital services are increasingly turning to large‑language models to streamline citizen interactions, and GDS’s recent pilot results provide a concrete proof point. By leveraging Anthropic’s Claude via Amazon Bedrock, GOV.UK Chat not only reached a 90% accuracy rate—outperforming many consumer‑focused chatbots—but also adhered strictly to official guidance, ensuring answers align with published policy. This level of precision is critical for public trust, especially when dealing with complex topics like tax or immigration, and signals that AI can be responsibly integrated into high‑stakes environments.

User experience data from the pilots underscores the delicate balance between speed and reliability. While the assistant’s average latency of 10.7 seconds is acceptable for most queries, respondents indicated higher satisfaction when answers arrived faster, prompting GDS to explore streaming responses and other latency‑reduction techniques. The rollout strategy—starting with the GOV.UK app before extending to the main website—allows the team to refine performance at scale, potentially reducing call‑center volumes and delivering more consistent, self‑service options for millions of citizens.

Safety remains a top priority, as evidenced by the 508 jailbreak attempts that were successfully neutralized by built‑in guardrails. GDS’s proactive monitoring and rapid mitigation framework illustrate a mature approach to AI risk management, a model other agencies are likely to emulate. With the ability to upgrade models on Amazon Bedrock, the platform is positioned for continuous improvement, ensuring that future iterations can maintain high accuracy while expanding capabilities such as personalized hand‑offs to human advisers. This initiative marks a significant step toward modernizing public‑sector digital services through trustworthy AI.