AI Can’t Even Forecast Inflation

Key Takeaways

• ChatGPT's inflation forecast error up to 12x Fed model.
• Errors persisted even after true out‑of‑sample period.
• Training data cutoff undermines genuine out‑of‑sample testing.
• Fed's Cleveland nowcast remains benchmark for macro forecasts.
• Results question AI's suitability for high‑stakes economic modeling.

Pulse Analysis

The Pentagon’s recent contract with OpenAI has reignited optimism that generative AI can power next‑generation weapons and decision‑support tools. At the same time, financial firms are racing to embed large language models into forecasting pipelines, hoping to outpace traditional econometric approaches. Yet, the value of any AI system hinges on its ability to predict real‑world variables accurately, especially those that drive policy and market moves such as inflation. When a model cannot reliably forecast a core macro indicator, its utility for high‑risk domains becomes questionable.

A team led by Huiyi Li at the San Francisco Fed put ChatGPT through a rigorous horse‑race against the Cleveland Fed’s nowcast model. Using only data that would have been available at the time of each forecast, they measured out‑of‑sample errors across three intervals: pre‑cutoff (up to April 2023), ambiguous (May 2023‑April 2024), and true out‑of‑sample (May 2024 onward). Across all periods ChatGPT’s root‑mean‑square error was dramatically higher—more than twice the Fed model before the cutoff, twelve times larger during the ambiguous window, and seven times larger after the cutoff. The study also highlighted that the model’s training data, which extends to April 2023/2024, contaminates any claim of genuine out‑of‑sample testing.

These findings send a clear signal to defense contractors, hedge funds, and enterprise software vendors: generic large‑language models are not a plug‑and‑play solution for macroeconomic prediction. Investors should temper expectations that AI will replace specialized econometric tools, and policymakers must scrutinize AI‑driven analytics before integrating them into critical decisions. Future research may improve performance by fine‑tuning models on high‑frequency economic data or by hybridizing AI with traditional statistical frameworks, but until then the Cleveland Fed’s nowcast remains the gold standard for inflation forecasting.