Greg K.SEO and analytics at Adcore · LinkedInThe llms.txt file briefly tells AI agents what's on a site and where to go for details. It doesn't affect visibility in Google: Google says outright that it ignores such files. Large studies also found no link to mentions in AI search. Still, some sites do benefit from it, and below we look at whether yours is one of them.
What is llms.txt and what does it look like
It's a Markdown file that lives at your-site/llms.txt. It says what the site is and lists pages with short explanations. It's meant to be read not by people but by agents: programs that look for answers or carry out tasks on someone's behalf.
The format was proposed by researcher Jeremy Howard in September 2024. His logic is simple: an agent has a hard time wading through menus, banners and scripts, while the site owner knows best what matters on the site. A second version of the specification came out in August 2026. It's still a proposal, not a standard. It says nothing about rankings or search visibility.
Only the first line with the name is required. The rest is optional:
# Site name > One sentence about what you do. Additional explanations, if needed. ## Services - [SEO audit](https://your-site/seo-audit): what we check and what you get - [Prices](https://your-site/prices): cost by type of work ## Optional - [Contacts](https://your-site/contacts)
You'll often see llms-full.txt mentioned alongside it. It isn't part of Howard's specification: this kind of file, with all the documentation text gathered in one place, was introduced by documentation platforms such as Mintlify.
Does llms.txt affect Google rankings
No. On December 3, 2025, an llms.txt file suddenly appeared in Google's documentation for webmasters, the very file Google had spent half a year saying wasn't needed. SEO specialist Lidia Infante asked Google's John Mueller whether this was an endorsement or whether he was trolling. He replied briefly: "hmmn :-/". A few hours later the file was gone and its URL returned a 404. It looks like the documentation platform had created it on its own.
Then Google answered officially, in its guide to optimizing for AI:
The same page says it's perfectly fine to create such a file for other systems: for visibility in Google, it will "neither harm nor help."
Mueller explained the reason very vividly on a Google podcast: "It's basically you're telling these systems, like, I have the best website ever." The file is written by the site owner, so a system that compares millions of sites can't rely on it. Google has been through this before with the meta keywords tag, where everyone wrote whatever they liked about themselves: in 2009 the company officially announced that it doesn't use it for ranking, and Mueller and Gary Illyes compare llms.txt to exactly that. It's the same logic as in E-E-A-T: what matters is not only what a site says about itself but also what others say about it.
One caveat. Lighthouse, Chrome's audit tool, checks for llms.txt in version 13.3, but in a separate experimental category for browser agents, not in SEO. There's no contradiction: Search and a browser agent do different jobs.
Do ChatGPT, Claude and Perplexity read llms.txt
Officially, nobody has promised that. We checked the help pages OpenAI, Anthropic and Perplexity publish about their bots: they describe in detail which bot does what, and llms.txt isn't mentioned as a source for search. We found no statements from Microsoft about this file either.
The largest study came from Ahrefs: 137,210 domains, traffic for May 2026.
The largest groups and all AI bots are shown. SEO audit includes Ahrefs' own bots. A request alone doesn't mean the file was used. Data: Ahrefs, May 2026.
A study by SE Ranking came at it from a different angle, comparing almost 300,000 domains: does AI cite sites with this file more often? It found no link. When the llms.txt feature was removed from the statistical model, the model became more accurate. That's a correlation, not proof there will never be an effect, but there's no reason to expect one right now.
Logs don't give a simple answer either. In June 2025 Mueller wrote that no chatbot requests this file. A month later, SEO specialist Ray Martinez showed logs in which an OpenAI bot fetches it every 15 minutes, and Ahrefs sees GPTBot in about 4.5% of requests. So bots do sometimes fetch the file. But requesting a file doesn't mean using it in an answer, and there's still no evidence of the latter.
We ran our own check too. We took 114 sites that rank in Ukrainian search results for the topics of our articles on marketing, websites and medicine, and requested /llms.txt on each one. A real file turned up on 20 of them. The other 94 sit in the same top results without it, so it clearly doesn't determine a site's place in the rankings.
Who actually needs llms.txt
Anyone whose agents need to get something done from documentation. The format's author himself writes that the file is used most for software documentation, where coding agents look up API descriptions through it. See where large companies keep their files: docs.stripe.com/llms.txt, developers.cloudflare.com/llms.txt, platform.claude.com/llms.txt. All of these are documentation domains for developers, not marketing sites.
| Your site | Worth it? | Why |
|---|---|---|
| API documentation, a software product for developers | Yes | Coding agents really do fetch these files |
| A service with a large help center | Yes, on the help domain | An agent finds the right instruction for the user faster |
| A service business, a clinic, a store, a blog | Not a priority | Google ignores it and no effect is visible in AI search. No harm if it takes you half an hour and you keep it up to date |
There's one more reason not to rush: the file has to be maintained. If pages change and the list doesn't, an agent will follow old links. For a service business, the same hours pay off more when spent on copy that answers customers' questions and on mentions on other sites. We cover where to start in our article what is SEO.
What really controls AI access to a site
robots.txt. Unlike llms.txt, its rules are described in the official help pages of the companies themselves, and search and model training are handled by different bots. So you can stay in AI search and still keep your content out of training:
User-agent: OAI-SearchBot Allow: / User-agent: Claude-SearchBot Allow: / User-agent: PerplexityBot Allow: / User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: /
| Bot | Why it visits | If blocked |
|---|---|---|
| OAI-SearchBot | Showing the site in ChatGPT search | The site drops out of ChatGPT search |
| Claude-SearchBot | Search quality in Claude | Less visibility in answers that use search |
| PerplexityBot | Showing and linking in Perplexity search | The site drops out of Perplexity search |
| GPTBot | Training OpenAI models | Content isn't used for training; search isn't affected |
| ClaudeBot | Training Anthropic models | Content isn't used for training |
Bots that visit at a specific person's request follow different rules: OpenAI says robots.txt rules may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores them. OpenAI and Perplexity pick up changes in about a day.
The most common problem is mundane. Someone once blocked all bots in robots.txt or at the server level, and the site disappears from AI search without a single error in any interface. Mueller put it bluntly: "don't block agents. I think that hurdle will be the biggest, for most sites." So the first step is simple: open your-site/robots.txt and check whether AI search bots are blocked there.
Want to know what exactly is holding your site back in search, including AI search? Start with an SEO audit from Adcore: we'll check bot access, content and mentions, and tell you plainly what's worth doing and what isn't.
How to create llms.txt: generator and plugins
If your site fits the table above, or you just want to have the file, grab it ready-made. Nothing is sent anywhere: the fields and the result stay in your browser.
Fill in the fields and the file builds as you type. Only the name is required.
# Adcore > A digital agency in Kyiv: SEO, paid search and website development for businesses in Ukraine. ## Services - [SEO audit](https://adcore.ua/en/services/seo-audit): what we check and what you get - [Paid search](https://adcore.ua/en/services/google-ads): setting up and running Google Ads - [Website development](https://adcore.ua/en/web-studio): stages, timelines, what goes into the price
The URLs in the example are real; replace them with your own.
The file goes in the root of the site so that it opens at your-site/llms.txt. Under the second version of the specification it can also go in a subfolder, and then it describes only that folder. The author suggests a simple test: give an agent only this file and ask a question about your site.
On WordPress, plugins can create the file. In Yoast SEO the feature has to be turned on separately, the file updates weekly and you can pick pages by hand. In Rank Math the file is empty until you set it up, and pages with noindex don't make it in.
What to check in an automatic file. This is our conclusion, not a Google requirement: a plugin doesn't know which pages matter most to you and may put an old post or a utility page at the top. Check what it pulled in. Ahrefs separately mentions the risk of hidden instructions for agents, so links to pages written by someone outside your team are better left out of the file.
llms.txt FAQ
Will llms.txt help me get into ChatGPT answers?
There's no evidence of it: no promises from OpenAI and no link in the studies. For ChatGPT search, what matters is that OAI-SearchBot can access your pages.
How is llms.txt different from robots.txt?
robots.txt tells bots where they may go. llms.txt tells an agent what to read when it already needs information. One file doesn't replace the other.
Does llms.txt replace a sitemap?
No. A sitemap lists pages for search engines, while llms.txt gives agents a short selection with explanations.
Can llms.txt harm a site?
Not in Google: the company says outright that the file will neither harm nor help. The harm is of another kind: hours spent on a file nobody reads.
How many sites already have llms.txt?
It depends on the sample: SE Ranking counted about 10%, Ahrefs 28%, but Ahrefs' customers skew toward more technically savvy owners.
Sources
Quotes are given as in the originals.
- The /llms.txt file, v2, Jeremy Howard, September 2024, updated August 10, 2026.
- Google's guide to optimizing for generative AI, Google Search Central, 2026.
- Google adds LLMs.txt to Search developer docs, Search Engine Roundtable, December 3, 2025.
- Search Off the Record podcast, Mueller's words, Search Engine Journal, June 2026.
- "Don't block agents," Mueller's words, Search Engine Journal, June 2, 2026.
- Google does not use the keywords meta tag in web ranking, Google Search Central Blog, 2009.
- Lighthouse and llms.txt, Search Engine Journal, May 2026.
- We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read, Ahrefs, June 15, 2026.
- LLMs.txt: Why Brands Rely On It and Why It Doesn't Work, SE Ranking.
- Google Says No AI System Currently Uses LLMs.txt, Search Engine Roundtable, June 2025.
- Ray Martinez's logs, Search Engine Roundtable, July 2025.
- Overview of OpenAI Crawlers, OpenAI.
- Anthropic's bots and how to manage them, Anthropic.
- Perplexity Crawlers, Perplexity.
- llms.txt in Yoast SEO, Yoast, June 10, 2025.
- llms.txt in Rank Math, Rank Math.