Glossary › llms.txt
What Is llms.txt?
llms.txt is a plain-text file placed at your site's root that tells AI crawlers — the bots used by tools like ChatGPT, Perplexity, and Anthropic's Claude — what content on your site is available, how it is organized, and how they are permitted to use it. It is the AI equivalent of robots.txt, but designed specifically for large language models.
Plain-English Definition
robots.txt tells Google's Googlebot which pages it can crawl. llms.txt does something similar but for AI systems: it provides a structured, human-readable file that describes your site's content, identifies your key pages, and optionally clarifies how AI tools may use or reference your content.
The format was proposed by Jeremy Howard in 2024 and has gained adoption as a lightweight standard. A typical llms.txt file includes a site description, links to key sections (blog, services, about), and optional licensing notes for AI usage. It is written in Markdown or plain text and lives at yoursite.com/llms.txt.
Why It Matters
AI search tools and LLMs crawl the web to build their knowledge bases. A well-structured llms.txt file makes it easier for those systems to understand your site's structure and identify your most authoritative content. It signals that your site is technically modern and AI-crawler-aware — which is increasingly a GEO signal.
It will not guarantee that an AI cites your content, but it removes friction for the crawlers that make those citation decisions. For businesses investing in content marketing, it is a low-effort addition with upside.
How to Create One
A minimal llms.txt includes your site name, a one-paragraph description of what your site covers, and a list of your most important URLs with brief descriptions. It does not require a developer — it is a plain text file you upload to your site's root. More advanced versions include content licensing terms, preferred citation formats, and explicit crawler permissions.
Need help with AI search optimization? Book a free call.
Book a Free Call