llms.txt is a plain Markdown file at the root of a website that lists, in a fixed format, the pages a large language model should read to understand the site. There is no evidence that it makes ChatGPT, Claude, Gemini or Perplexity cite you more often, so treat it as cheap housekeeping rather than a visibility lever.
The rest of this page covers what the file contains, who reads it, what Google has said about it, why it can still be worth publishing, and what actually decides whether an assistant names your business.
What exactly is llms.txt?
The idea comes from Jeremy Howard of Answer.AI. In his original proposal, site authors write a Markdown file listing the content an LLM should use, and optionally serve .md versions of their pages so a model can read clean text instead of parsing HTML, navigation and scripts. The post is explicit that this is a proposal for how authors can outline LLM-friendly context, not a prescriptive standard.
The specification at llmstxt.org defines the format. The file sits at the site root (or a subpath) and must begin with an H1 containing the project name, followed by a blockquote summary. After that come optional sections with lists of links. A minimal file has this shape, with your own page addresses in place of the placeholders:
```
Example Freight Software
Software that measures pallet and parcel dimensions for warehouses and carriers.
Product
- How dimensioning works: the measurement process
- Pricing: plans and what they include
Guides
- Choosing a dimensioning system: buyer checklist ```
The specification remains open for community input on GitHub. It has not been ratified by any standards body, and no AI company is bound by it. Community directories such as llmstxt.site, directory.llmstxt.cloud and llmstxthub.com catalogue sites that have adopted it.
Who actually reads an llms.txt file?
The proposal was written with a specific use in mind: a model or a developer tool that has been pointed at a website and needs a fast map of what matters there. Think of a developer pasting a documentation URL into a coding assistant. In that situation a curated list of clean pages is useful.
That is different from how assistants answer a buyer's question. When someone asks ChatGPT or Perplexity which supplier to choose, the assistant either answers from its trained knowledge or runs a web search and fetches the pages that look most relevant. None of the sources above shows a major assistant committing to read llms.txt as part of that process.
What we see in server logs is traffic to ordinary pages. Across the retained nginx logs of an edtech platform we work with (pickaclass.com), from 28 July to 11 August 2026, ClaudeBot made 19,272,125 requests, Meta-ExternalAgent 3,854,076, GPTBot 2,744,985, Applebot 2,655,555, Amazonbot 1,170,268 and Googlebot 52,292. In the same fortnight, live retrieval agents (ChatGPT-User, OAI-SearchBot, Perplexity-User), which fetch a page while an assistant is answering a real person, made 1,918 fetches on that platform's content pages. A client blog of a few dozen articles received 56 such fetches across 24 different articles.
The practical point: assistants fetch the page that answers the question. If you want to know whether anything requests your llms.txt at all, check your own logs for that path. Our guide on finding AI crawlers in server logs shows how.
What does Google say about llms.txt?
Google has been direct. As Search Engine Roundtable reported, Google updated its help documentation on generative AI in Search to say that site owners don't need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search or its generative AI features, because Google Search doesn't use them.
So for AI Overviews and Google's other AI features, an llms.txt file is neither a requirement nor a ranking signal. Google crawls, indexes and ranks your normal pages. The same report frames it as a file that won't help or hurt, which matters for the next question.
Is there any evidence that llms.txt gets you cited?
The largest public test points the other way. An SE Ranking analysis of roughly 300,000 domains, covered by Search Engine Journal, found that only 10.13% of domains had the file and that there was no statistical correlation between having it and how often a domain was cited by AI. Their prediction model became more accurate once the llms.txt variable was removed.
Our own measurements point to the same conclusion from a different angle: tidy machine-readable files do not compensate for a small or hard-to-find site.
- A site with complete markup (server rendering, Article, FAQPage and Organization JSON-LD, a sitemap) but only 75 URLs was still absent from answers on its own article's exact topic. A site with about 190,000 crawlable URLs was crawled constantly.
- On 19 October 2026 we asked assistants for the exact titles of 18 articles, each older than three weeks, published on three client sites. ChatGPT found 1 of 18; Claude found 7 of 18. Google had indexed 29 of 30 articles of one of those sites. Being in Google's index did not mean an assistant could find the page.
- In a test on 23 September 2026 of 20 German business questions, the model answered 10 without searching at all and cited nobody. Where it searched, it cited primary sources such as laws, ministries, the OECD and large accounting firms on 9 of 10. Firms were named only on "who should I hire" or "how to choose" questions.
When half the questions are answered without a search, no file on your server is consulted. When a search does happen, what gets cited is the page that best answers the question, often a primary source or a well-established site.
Why publish one anyway?
Because it is nearly free, and it has a few real uses.
It costs almost nothing. On WordPress, Yoast SEO's llms.txt feature generates the file at your root directory and updates it weekly once enabled. On other platforms, a hand-written file is short and simple: a title, a one-line summary and a list of links.
It helps where people point tools at your site. If you sell software with documentation, or your buyers are developers, some of them will paste your URLs into coding assistants and agents. A clean map of your key pages, plus Markdown versions as the original proposal suggests, makes that easier.
It forces a useful decision. To write the file you have to choose the pages that best explain what you do, for whom, and at what price. In doing so you may find that the page answering your buyers' main question does not exist yet. That gap matters far more than the file.
Google says it won't hurt. There is no penalty for having one.
What it should not be is the project. If anyone sells llms.txt as the thing that will get you into AI answers, the evidence above does not support that.
What should you fix before worrying about llms.txt?
The things that decide whether an assistant can find and use your pages at all. Here is how llms.txt compares with the files and channels that do affect access:
| Item | What it does | Who uses it | Effect on AI answers |
|---|---|---|---|
| robots.txt | Allows or blocks crawlers by path | Well-behaved search and AI crawlers are expected to honour it | A wrong rule can remove you entirely |
| XML sitemap | Lists every URL you want crawled | Google, Bing and other crawlers | Helps new pages get discovered and indexed |
| Bing indexing and IndexNow | Gets pages into Bing's index quickly | Bing, and assistants that search through it | Pages missing from the index can't be found by those searches |
| Pages answering buyer questions | The content that actually gets fetched and cited | Live retrieval agents and search crawlers | The main thing assistants cite |
| llms.txt | Curated Markdown map of key pages | Some developer tools; Google says Search doesn't use it | No measured effect on citations |
robots.txt deserves particular attention because mistakes there are silent. On our own site, a rule meant to hide a single redirect page (/b) was written as a prefix. Because robots.txt rules match prefixes, from 11 August to 21 October 2026 it also closed /blog, /business and every article in our business blog to all crawlers. Bing Webmaster Tools listed our business page as "discovered but not crawled" from 11 August 2026, and Google had none of the business pages indexed. We changed the rule to /b$ on 21 October 2026. An llms.txt file listing those pages would have changed nothing: the crawlers were not allowed in. Our guide to allowing GPTBot and other AI crawlers in robots.txt covers the rules, and how AI crawlers find a website's pages explains discovery in more detail.
After access, the work is content and measurement. Write pages that answer the questions your buyers actually ask, especially "who should I choose" questions, since those are where assistants name businesses. Then measure, carefully: asked the same question three times, an assistant often names different businesses. We count a business as named for a question if any of three asks names it, and compare weeks only on the same questions and the same number of asks.
If you want a starting point, the free check at https://www.itgrows.ai/ generates five buyer questions from your homepage and shows in 25 to 40 seconds whether ChatGPT and Perplexity name you. ItGrows also publishes articles on clients' own domains, fixes crawler access and indexing, and each week puts the client's buyer questions to ChatGPT, Claude, Gemini and Perplexity three times each, for $499 a month (plus a one-time $499 setup, free with 3 months or a year). Nobody can guarantee that an assistant will name a business, and we don't.
Frequently Asked Questions
Does llms.txt replace robots.txt?
No. robots.txt tells crawlers which paths they may access, and well-behaved crawlers are expected to honour it. llms.txt is only a suggested reading list; it cannot grant access, and a page blocked in robots.txt stays blocked whatever llms.txt says.
Will Google penalise my site for having an llms.txt file?
No. Google's documentation says Search doesn't use such files, and reporting on that update describes llms.txt as neither helping nor hurting rankings. It is simply ignored by Google Search and its AI features.
Should I also serve Markdown versions of my pages?
The original Answer.AI proposal suggests it, and it can help developer tools read your content cleanly. For a typical small business website it is optional; well-structured, server-rendered HTML pages that answer buyer questions matter more.
How can I tell whether llms.txt is making any difference?
Check your server logs for requests to /llms.txt to see whether anything fetches it at all. Then track whether assistants name you on a fixed set of buyer questions over several weeks, asking each question more than once, since answers vary from run to run.