How to Measure Sophisticated AI Use at Work

Why It Matters

It shows that AI adoption alone yields limited value; scaling the identified behaviors can dramatically boost productivity and strategic outcomes across enterprises.

Key Takeaways

• Sophisticated users iterate, frame problems, choose tools intentionally
• Only 5% of users exhibit high-impact AI behaviors
• Thirty behavioral signals predict effective human‑AI collaboration
• KPMG created training, playbooks, and AI‑First behavior framework
• Findings published in Harvard Business Review guide enterprises

Pulse Analysis

The joint KPMG‑McCombs investigation examined 1.4 million AI‑driven interactions across the firm’s back‑office, producing the first large‑scale, data‑backed map of what separates routine prompting from high‑impact collaboration. Published in the Harvard Business Review, the research shows that sheer access to generative models does not automatically translate into productivity gains. Instead, a small cohort—about five percent of users—demonstrated markedly better outcomes by treating the model as a reasoning partner rather than a simple query engine. This insight gives executives a measurable benchmark for AI maturity.

The study distilled roughly 30 observable signals into a practical blueprint for sophisticated use. Key behaviors include returning to the same model repeatedly, persistently refining outputs, launching ambitious initial requests, and deliberately selecting the most suitable tool or model. Users also frame problems explicitly, assign roles to the AI, and demand explanations of its reasoning. These patterns create a feedback loop where iteration fuels ambition, and ambition drives strategic tool choice, reinforcing engagement. By quantifying these actions, organizations can spot high‑potential employees, coach peers, and embed the behaviors into performance metrics.

KPMG has already turned the findings into an “AI‑First” learning ecosystem, complete with the aIQ Learning Academy, role‑based playbooks, and peer‑led champion networks. The firm’s rollout demonstrates how data‑driven coaching can shift AI from a peripheral perk to a core workforce capability, accelerating client‑facing services and internal efficiency. As more companies grapple with the hype‑vs‑value gap in generative AI, the KPMG model offers a replicable path: identify behavioral signals, institutionalize iterative practices, and scale them through structured training. The result is a more resilient, AI‑augmented talent pool ready for complex problem‑solving.