Marketers have been told to implement LLMs.txt to win visibility in AI search. The data says otherwise. So does Google.
LLMs.txt is a proposed standard, not an adopted one.
No major LLM parses LLMs.txt for discovery.
Fewer than 3% of LLMs.txt files receive traffic from AI bots.
Adding LLMs.txt can’t hurt, but it shouldn’t be the focal point of a visibility strategy.
Executive Summary: LLMs.txt is a proposed standard, designed to improve AI crawl efficiency – not ranking or discovery. No AI model requires it to parse the web content, which is why it’s rarely accessed and, even then, use cases are few and extremely narrow. The apparent Google Search vs. Chrome Lighthouse contradiction dissolves once you separate the two ideas: discovery vs. operability – LLMs.txt helps with neither.
When LLMs.txt was proposed as a standard, many marketers jumped the gun, thinking it the “next big AI visibility hack”. As it turns out, it’s anything but.
The evidence shows LLMs.txt carry no proven search benefit, near-zero real-world consumption, no indication of universal acceptance.
Yet, many continue to slot it into their AI search SEO, as if hoping that a single file will become the silver bullet it was suggested to be.
It won’t.
LLMs.txt is a community-proposed web standard, designed specifically to help Large Language Models (LLMs) and emerging AI agents crawl the website and index the most valuable content as efficiently as possible.
The idea is simple: a single Markdown file at the domain root that points crawlers toward the site’s most context-rich pages, such as API docs, return policies, product taxonomies, and similar items you don’t want AI to miss.
The concept is nothing new. It borrows heavily from robots.txt and sitemap.xml, both of which have been doing essentially the same thing for over a decade – telling traditional bots where they are allowed to go and what content they should index. LLMs.txt only adapts it for the AI era.
However, the crucial word in the previous definition is “proposed,” as the concept still exists almost exclusively on llmstxt.org as an idea, not as something AI providers or the community have agreed to follow.
Yes, but actually, no. A recent deep dive by Ahrefs discovered that 28% of sites publish LLMs.txt documents, but over 97% of them receive zero traffic from any source, and of the <3% requests that did happen, AI bots were only ~1%. In addition, no major provider agreed to use LLMs.txt to discover or rank the content – not OpenAI, not Anthropic, not Meta, and not even Google, despite it sending mixed signals.
Ahrefs · 137,000 domains · May 2026
High creation. Near-zero consumption.
Source: Ahrefs analysis of 137,000 domains, May 2026.
In May 2026, Chrome’s Lighthouse 13.3 added an “Agentic Browsing” audit that checks if your website has LLMs.txt. Only days later, Google’s AI optimization guide listed LLMs.txt as the first thing you don’t need. Cue the confusion.
Because it’s two different teams answering two different questions – discovery vs. operability. So the contradiction is mostly a misread, and the fix is one distinction.
Put simply, LLMs.txt has nothing to do with either. They are essentially a lightweight, high-level roadmap designed to prevent AI from burning through tokens while reading a website – after it already knows the website exists.
In fact, Google’s audit documentation characterizes LLMs.txt as an “emerging convention” without which “agents may spend more time crawling the site to understand its high-level structure and primary content.”
In a recent Search Off the Record podcast, Google’s John Mueller pointed out inherent untrustworthiness as a critical flaw of LLMs.txt. Because site owners are the ones who write the file, they own the entire narrative: they can say their website is the best, their pages are valuable, their product is a must-buy. However, the data on the website itself may not reflect that narrative.
This is where Mueller arrives at the critical insight: AI models can read website data directly – so why would they trust LLMs.txt over the actual content? Or, as he himself put it: “So in an LLM system, it… basically, by design, can’t trust what is here [in LLMs.txt] as a way of differentiating between different websites.”
As a core facet of your AI Search optimization strategy – no, not really. There’s simply no evidence that LLMs.txt aids ranking or discovery, and since virtually every major LLM provider declined to commit to it, it’s reasonable to assume it won’t become the “gold standard” anytime soon (or ever).
As a near-free experiment – sure. LLMs.txt is cheap to create and could carry a small early-mover advantage, IF it ever gets standardized. Plus, implementing it is low risk, as long as you make sure the file doesn’t return a server error or hit a login wall if an AI agent does come knocking. However, that’s about the strongest case to be made for it.
Tina Clarke is the AI SEO Manager at ZeroClick Labs, specializing in AI search optimization and Generative Engine Optimization (GEO). With a strong foundation in content strategy, technical SEO, and operations, she leverages her expertise to help brands shift from traditional rankings to discoverability and excel in AI-driven ecosystems.
It’s about strategy
Being cited by AI Overviews, ChatGPT, Perplexity, or any other major LLM never did – and never will – depend on a single file.
What it does depend on is understanding the signals behind citations – and leveraging them to engineer a repeatable, scalable AI SEO strategy that compounds visibility over time.
That’s the work ZeroClick Labs does.
“Our agency had no idea how to approach AI visibility. ZeroClick only does this one thing so they actually know what works. Worth every penny just to not waste time figuring it out ourselves.” – Jay