llms.txt, explained straight
What the file actually does, who demonstrably reads it, and where its limits are — with a free validator that grades yours and an always-fresh alternative for the facts a static file cannot keep true.
What llms.txt is
llms.txt is an optional convention: a markdown file at the root of your domain that tells AI systems what your site is and where its key pages live. The expected shape is deliberately small — one # H1 title, an optional > blockquote summary, then ## sections of named links with one-line descriptions. Its companion llms-full.txt inlines full reference content for tools that want everything in one fetch.
An index, not a directive
The file points AI systems at your key pages. It does not make what they find — or what they already believe about you — accurate, and it binds no model to read it.
Who demonstrably reads it
Coding agents and IDE assistants fetch llms.txt and llms-full.txt to learn an API or docs site before writing integration code. That use case is real and measurable.
What is unproven
Whether consumer AI engines fetch the file when answering everyday questions. No major model provider has committed to reading one in production — publish for the use case that works, not the one you hope for.
Its failure mode is staleness
A hand-edited file rots the day your site changes, and nothing tells you it rotted. If you publish one, generate it from the same source of truth that serves your site.
Check yours in about ten seconds
The free llms.txt validator fetches your file, checks AI crawlers can reach it, parses the structure, samples the links, and hands back a letter grade with the exact fixes. Keyless, no signup — and a missing file is reported as the neutral state it is, without scare copy.
How Entidex serves its own
We publish both halves of the convention and hold them to the standard the file argues for: entidex.com/llms.txt is the concise index, and entidex.com/llms-full.txt is the expanded developer reference — generated at request time from the live API specification and tool registry, so it cannot drift from the served API. Hand-copying an API surface into a document published for machine consumption is precisely the staleness failure the convention is prone to.
The file is an index. The record is the product.
For entity facts — who you are, what is verifiably true, what AI engines currently get wrong — a text file is an unsigned assertion anyone could author. The Entity Knowledge Statement is the signed, verified record with per-fact provenance, rebuilt from live data on every fetch; the grounding embed serves it on your own domain, including as a ready-made llms.txt block that never goes stale.
Go deeper with Entidex
The full entity intelligence platform — beyond Explore Entidex
- Continuous multi-source entity observation
- Alerts when sources diverge or drift
- Cross-surface consensus + visibility over time
- Evidence-anchored intelligence reports
Frequently asked questions
What is the difference between llms.txt and llms-full.txt?
llms.txt is the concise index: one H1 title, an optional blockquote summary, then H2 sections of named links with one-line descriptions — a map AI systems can fetch in a single small request. llms-full.txt is the expanded companion that inlines the full reference content behind those links, for tools that want everything in one fetch instead of following links. A site can serve either or both; the index should stay small and the full file carries the depth.
How do I create an llms.txt file?
Serve a markdown file at /llms.txt on your domain: start with a single # H1 naming the site, add a > blockquote summarising what it is, then ## sections whose bullet lists carry [name](url) links with one-line descriptions of your key pages. Keep it small enough to sit in a prompt context. If your API or docs change often, generate the file from the same source of truth that serves them — a hand-edited copy goes stale the day your site changes.
Does llms.txt help SEO or GEO?
Honestly: adoption is ahead of consumption. The documented, working use case is coding agents and IDE assistants fetching the file to learn a docs site or API before writing integration code. There is no solid public evidence that the consumer engines behind ChatGPT, Claude, Gemini, Perplexity and Grok routinely fetch llms.txt when answering everyday questions, and no major model provider has committed to reading one in production. Publish it for the use case that works; do not expect it to change how AI describes your brand.
Where should the file live, and can crawlers reach it?
At the domain root — /llms.txt — with /.well-known/llms.txt as a recognised fallback, served as plain text or markdown with an HTTP 200. Two common failures: the app shell or a 404 page served with a 200 status (HTML where markdown should be), and a robots.txt that blocks AI crawler agents from the very file published for them. The free validator on this site checks both.
What is the alternative for facts that must never go stale?
For the entity facts that matter — who you are, what is verifiably true about you, and what AI engines currently get wrong — Entidex publishes a signed Entity Knowledge Statement: a verified record with per-fact provenance, rebuilt from live data on every fetch. Customers can serve it on their own domain through the grounding embed, including as a ready-made llms.txt block that updates when the record updates. A static file cannot do that.