Does llms.txt Matter? We Tracked 10 Sites to Find Out

Why It Matters

Businesses should not allocate SEO resources to llms.txt expecting traffic gains; focus on content that AI models can actually surface.

Key Takeaways

• llms.txt adoption did not boost AI traffic in most cases.
• Growth linked to new functional assets and media coverage.
• No major LLM provider currently parses llms.txt files.
• Token efficiency benefits mainly developer documentation, not ecommerce.
• Treat llms.txt as infrastructure, not a growth strategy.

Pulse Analysis

The llms.txt specification emerged as a proposed standard for AI agents to discover high‑value site content, mirroring the familiar sitemap format. Early enthusiasm grew when Google briefly rolled the file out across its developer properties, suggesting official endorsement. However, the rapid removal and John Mueller’s clarification that the files were not intentionally exposed highlighted the uncertainty surrounding its practical adoption. This back‑and‑forth has left marketers questioning whether the effort of maintaining an llms.txt file is justified.

Empirical evidence from a 90‑day before‑and‑after analysis of ten sites across finance, SaaS, ecommerce, insurance, and pet care paints a clearer picture. The two sites that saw AI‑driven traffic increases were simultaneously launching high‑impact PR campaigns, restructuring product pages with extractable data tables, and publishing downloadable templates that solved concrete user problems. Those initiatives, not the presence of llms.txt, drove the spikes. Moreover, token‑efficiency arguments—where a concise markdown file reduces parsing costs—primarily benefit developer‑focused documentation, a niche not shared by most consumer‑oriented sites.

For most businesses, llms.txt should be treated as a technical convenience rather than a ranking signal. Implementing the file can aid AI agents that eventually adopt the standard, but the immediate ROI lies elsewhere: creating functional, extractable assets; fixing crawl errors; optimizing content for user intent; and earning authoritative backlinks or media coverage. Allocating resources to these proven tactics will generate measurable traffic and brand visibility, while llms.txt remains a low‑risk, low‑reward addition to the SEO toolkit.