llms.txt Explained: Does Your Website Actually Need One?
llms.txt is a plain-text file, placed at the root of your domain (yourdomain.com/llms.txt), that gives AI systems a short, curated, human-written summary of what your site is and which pages matter most. It is the AI-era equivalent of a sitemap written for a language model instead of a crawler — a welcome note rather than a technical index.
What llms.txt actually contains
A typical llms.txt is short: a one-line description of the site, a few sections grouping important pages by topic, and short link-with-description entries pointing to the pages an AI is most likely to find useful. It is written in plain Markdown, meant to be read directly, not parsed by a rigid schema.
What it does not do
llms.txt is not a robots directive — it has no power to block or allow crawler access; robots.txt still governs that entirely. It is also not (yet) formally adopted or consistently used by any major AI lab as an official input to their systems. Treat it as a low-cost bet on an emerging convention, not a guaranteed lever.
Is it worth adding?
Yes, with realistic expectations. It typically takes under an hour to write, costs nothing to host, cannot hurt anything, and gives any AI system that does choose to read it a much cleaner starting point than trying to infer your site's structure from raw HTML. Given the low cost, most sites should add one; just do not expect it to single-handedly fix a site that is blocked in robots.txt or written in dense, unclear prose.
Frequently asked questions
Where exactly do I put llms.txt?
At the root of your domain, exactly like robots.txt — for example, https://yoursite.com/llms.txt.
Does adding llms.txt guarantee AI systems will use it?
No. It is a voluntary convention, not a required or universally-honored standard. Some systems may read it, others may ignore it entirely today. Its value is asymmetric: near-zero cost, plausible upside.
Should I duplicate my whole site into llms.txt?
No — keep it short and curated. A long, unstructured dump of every page defeats the purpose, which is to give an AI a quick, human-edited orientation, not a second sitemap.