Is your llms.txt actually working?
Paste a domain. We fetch its llms.txt, check that AI crawlers can reach it, parse the structure, test the links, and hand back a letter grade with the exact fixes — in about ten seconds.
Free · no signup · checks /llms.txt and /.well-known/llms.txt
What is llms.txt?
llms.txt is an optional convention: a markdown file at the root of your domain that tells AI systems what your site is and where its key pages live. The expected shape is deliberately small — one # H1 title, an optional > blockquote summary, then ## sections of named links with one-line descriptions. Its companion llms-full.txt inlines full reference content for tools that want everything in one fetch.
A word of honesty most llms.txt tooling skips: adoption is ahead of consumption. The documented, working use case is coding agents and IDE assistants fetching the file to learn a docs site or API before writing integration code. Whether the consumer AI engines fetch it when answering everyday questions is unproven — so this validator grades what is measurable and claims nothing it cannot see.
What the validator checks
Exists and is served as text
We request /llms.txt and the /.well-known/llms.txt fallback, expect a plain HTTP 200, and catch the classic failure: an app shell or 404 page served with a 200 status — HTML where markdown should be.
Reachable by AI crawlers
We read your robots.txt and check whether the llms.txt path is blocked for the named AI crawler agents (GPTBot, ClaudeBot, PerplexityBot and their search/live siblings) — a blocked context file cannot be fetched by the crawlers it exists for.
Structure parses
H1 title, blockquote summary, H2 sections, and [name](url) link syntax — the shape parsers expect. Malformed link lines and bare URLs are flagged with the exact lines to fix.
Links are alive, size is sane
We sample up to ten linked URLs and check they respond, flag files too large to serve as prompt context, and report freshness signals like the Last-Modified header.
The file is an index. The record is the product.
An llms.txt tells AI systems where to look. It does not make what they find — or what they already believe — accurate. The failure mode is staleness: a hand-edited file rots the day your site changes, and nothing tells you it rotted.
Entidex approaches the same problem from the data side. The Entity Knowledge Statement is a signed, verified record of an entity’s facts — per-fact provenance, corrections to the claims AI engines currently get wrong, rebuilt from live data on every fetch. Customers can publish it on their own domain through the grounding embed, including as a ready-made llms.txt block that never goes stale — the always-fresh alternative to a hand-maintained file.
Go deeper with Entidex
The full entity intelligence platform — beyond Explore Entidex
- Continuous multi-source entity observation
- Alerts when sources diverge or drift
- Cross-surface consensus + visibility over time
- Evidence-anchored intelligence reports
Frequently asked questions
What is llms.txt?
llms.txt is an optional convention (proposed at llmstxt.org) for a markdown file served at /llms.txt that tells AI systems what a website is and where its key pages live. The expected shape is small: one H1 title, an optional blockquote summary, then H2 sections whose bullet lists carry named links with one-line descriptions. Its companion, llms-full.txt, is the expanded version that inlines full reference content for tools that want everything in one fetch.
Do AI engines actually read llms.txt?
Honestly: adoption is ahead of consumption. The clearly documented working use case is coding agents and IDE assistants fetching llms.txt or llms-full.txt to learn an API or docs site before writing code against it. There is no solid public evidence that the consumer engines behind ChatGPT, Claude, Gemini, Perplexity and Grok routinely fetch llms.txt when answering everyday questions — which is why this validator reports what is measurable (existence, fetchability, structure, link health) and does not promise a ranking effect.
What does the validator check?
Seven things, each with a concrete fix when it fails: (1) the file exists at /llms.txt or /.well-known/llms.txt; (2) it is served with HTTP 200 as text — not your app shell or 404 page; (3) the structure parses: an H1 title, a blockquote summary, H2 sections with markdown links; (4) a sample of up to ten linked URLs actually respond; (5) the file size is realistic for prompt context; (6) robots.txt does not block AI crawlers from fetching it; (7) freshness signals like the Last-Modified header. The report card rolls those into a letter grade.
Do I need an llms.txt file?
Not necessarily. It is an optional convention, and a missing file is a neutral state — this tool reports "not found" plainly, without scare copy. If your site is documentation, an API, or anything coding agents integrate with, a well-maintained llms.txt is cheap and demonstrably used. If your concern is how AI describes your brand or business, the file is a weak lever on its own: what matters more is whether the claims AI engines make about you are accurate — which is what an Entidex scan measures.
What is the alternative to hand-maintaining an llms.txt?
The failure mode of llms.txt is staleness: it is a hand-edited file that rots the day your site changes. Entidex publishes a signed, always-fresh alternative for the entity facts that matter: the Entity Knowledge Statement — a verified record with per-fact provenance, rebuilt from live data on every fetch — and a customer grounding embed that can serve that record on your own domain, including as a ready-made llms.txt block. The generated block updates when the record updates; a static file cannot.
Is the validator free, and what happens to the domain I enter?
Free and keyless — no signup, no account. The domain you enter is used to fetch that site’s public llms.txt, robots.txt and a sample of linked pages, and for nothing else. Checks are rate-limited per visitor, only public hosts are fetched, and no credentials or cookies are ever sent.