llms.txt: what it is, whether it does anything yet, and how to write one

llms.txt is a markdown file at /llms.txt that lists your most important pages in plain language, so an assistant reading your site knows where the good content is. It was proposed in September 2024. No major AI operator has published documentation confirming their assistant fetches it during retrieval. It is cheap enough to be worth publishing anyway, and this article explains why that is a reasonable position rather than a contradiction.

Note, 13 August 2026. The proposal moved to v2 on 10 August 2026, after this article was written. The argument below is unchanged, but the mechanics are not: v2 adds link relations for discovering a page’s markdown version, allows both .md URL forms, defines what a file at a subpath covers, and removes the mechanical meaning of the Optional section. Those changes are set out on our llms.txt reference page, alongside measured adoption across 1,048 sites. If you want files to copy rather than an argument, see llms.txt examples from real sites.

What the file actually is

A plain markdown document served as text/plain or text/markdown at the root of your domain. The proposed structure is short:

• An H1 with the name of the site or project.
• A blockquote directly under it, one or two sentences summarising what this is.
• H2 sections, each containing a markdown list of links, ideally with a sentence of context per link.
• Optionally a section named Optional, holding links that can be skipped when context is tight.

There is a companion convention, llms-full.txt, which holds the full text of your documentation rather than links to it. That one is aimed squarely at developer documentation and is overkill for most businesses.

The format is deliberately close to the shape a language model handles best: headings, short declaratives, links with context. That is not an accident and it is the most defensible thing about the proposal.

The honest state of adoption

Two things are true.

Documentation platforms and developer tools have adopted it widely. Plenty of technical companies publish one, and the tooling to generate it exists in most documentation frameworks.

Meanwhile no operator of a major assistant has published documentation saying their retrieval pipeline requests /llms.txt. Search representatives have been publicly sceptical, comparing it to the keywords meta tag, a file that sites filled in for years while nothing consumed it. Nobody has produced a controlled test showing a citation that would not have happened otherwise.

Every article that presents llms.txt as settled fact is either guessing or repeating somebody else who guessed. We sell audit honesty, so we will say plainly: the mechanism is unproven and the upside is speculative.

Why publish one anyway

The argument is not about evidence. It is about asymmetry.

The file is a few hundred bytes. Writing it takes an hour, most of which is deciding which of your pages actually matter. The maintenance cost is zero if you generate it. If it turns out nothing reads it, you have lost an hour and gained a written list of the pages you consider important, which is a useful artefact regardless.

If it does get read, it is read at the exact moment an assistant is deciding which of your pages to quote and how to describe what you sell. That is the highest leverage moment in this entire category. Being quoted at a price you do not charge, or described as something you are not, is a support ticket from somebody who is right, and it happens more often than most companies realise.

An hour against that is not a close call. It is also, importantly, the last thing to do rather than the first. If AI crawlers are blocked in your robots.txt, or your content only appears after JavaScript runs, llms.txt is decoration on a locked door. Fix those first.

Our own file, and why each part is there

We publish llms.txt and it is generated from the same configuration our pricing pages render from, which is the single most important decision in it. A hand written copy of a price list is a copy that is wrong the first time a price moves.

It opens like this:

# Visibility100x

> SEO and AI visibility for US startups and companies. Two things are sold
> separately: a platform for running audits, rank tracking and AI visibility
> checks yourself from $24 a month, and service plans from $99 a month where our
> team ships the fixes and publishes the content on your site.

One heading, one blockquote, and the two things we sell with their entry prices in the first forty words. If an assistant quotes nothing else, that sentence alone is accurate and complete.

Then the plans, each with its price and its limits:

## Platform plans, the software

Priced per month or per year. Every limit below is enforced by the server, not
advertised loosely.

### Individual

Price: $24 a month, or $249 a year.

- 5 sites, 50 pages a crawl
- 25 keywords tracked weekly
- 10 AI questions, 4 runs a month

Every figure sits next to the thing it prices. There is no sentence a machine has to unpick to work out which number applies to what. This is the part people get wrong most often: they write marketing prose in the file and then wonder why they get summarised badly.

Then a section of flat statements written specifically to be lifted:

## Facts worth quoting

- The free audit needs a free account and one email address, no card and no
  trial, and works on any URL.
- Every plan covers classic SEO and generative engine optimisation together,
  not one or the other.
- Every plan is month to month. There is no annual lock in and no trial on any
  plan.

And a closing note addressed to the reader we actually expect:

## Notes for AI systems

Prices on this page are list prices checked on 28 July 2026. The live figure any
customer is charged is always the one shown at /pricing.

That last section is the one worth stealing. It gives a date, it points at the authoritative source, and it tells a system what to do when the two disagree.

How llms.txt relates to the files you already have

File	Purpose	Read by	Enforced
robots.txt	Who may fetch what	Every crawler	Honoured by convention
sitemap.xml	Every URL you want indexed	Search engines	Yes, as a discovery input
llms.txt	Which pages are worth reading, and what you are	Unconfirmed	No

They do not overlap and none replaces another. robots.txt is permission, sitemap.xml is inventory, llms.txt is editorial judgement. The third one is the only one where you say what matters rather than what exists.

The mistakes worth avoiding

Writing marketing copy. The file is read by a machine deciding how to describe you. Adjectives cost you precision and buy nothing. Write the sentence you would be happy to see pasted into an answer verbatim.

Listing everything. A file that links to all four hundred of your pages has made no editorial decision and is worth exactly as much as your sitemap. Twenty links with a sentence each is more useful than four hundred bare URLs.

Letting it go stale. An llms.txt with last year’s prices is worse than no file at all, because it is confidently wrong at the moment of citation. Generate it from the same source of truth your product pages use, or put a reminder in your calendar and accept that you will ignore it.

Publishing it while blocking the crawlers. We see this more than you would expect. Check both in the same sitting.

Check yours

Our llms.txt checker fetches the file from your live domain and reports whether a real one comes back, alongside your robots.txt and sitemap status, because those three answer the discovery question together. It is worth noting that a server which returns your homepage for every unknown path will look like it has an llms.txt when it does not, so a check that reads the response rather than the status code is not a pedantic distinction.

If you want the format itself rather than the argument, the llms.txt reference has the v2 specification in full, and llms.txt examples has eleven real files from live domains with a template you can copy.

If the file is missing, treat it as an hour of work to schedule rather than an emergency. If your AI crawlers are blocked or your content does not survive without JavaScript, those are the emergency, and they are the two things worth fixing before you write a single line of markdown.

Sources

• llmstxt.org, the proposal itself, including the format specification quoted above.
• OpenAI, bots and user agents, for what OpenAI does document about fetching, which is a useful contrast: the crawler behaviour is written down and llms.txt is not mentioned.
• IETF, RFC 9309: Robots Exclusion Protocol, for the difference between a file with a standard behind it and a file with a convention behind it.
• Our own llms.txt, which is the worked example above and is regenerated on every deploy.