llms.txt File: The New Standard for AI Search Visibility (2026 Guide)

An llms.txt file is a curated, Markdown-formatted index that tells AI systems what matters most on your website — and adding one should take less than an hour.

If you have been following the shift from traditional search to AI-driven answer engines, you have likely seen the term show up alongside generative engine optimization and agentic AI optimization. The llms.txt standard was proposed by Jeremy Howard of Answer.AI on September 3, 2024, and published at llmstxt.org. It addresses a real problem: large language models have context windows measured in thousands of tokens, not millions of HTML pages, so they benefit from a curated summary of your site rather than a full crawl.

This guide covers what llms.txt is, how it differs from robots.txt, a full working example you can adapt, which AI systems currently read it, and an honest assessment of whether it moves the needle on rankings or citations today.

I run Verlua, a Sacramento web development agency. We have deployed llms.txt files for every client site we have launched since late 2024, and the format has become part of our default build checklist.

What Is an llms.txt File?

llms.txt is a plain-text file written in Markdown and placed at the root of a website (example.com/llms.txt). It provides a human-readable, machine-friendly summary of what the site is about and links to the most useful resources for answering questions about it. The format is intentionally minimal — the entire spec fits on a single page.

The rationale is practical. Modern web pages are bloated with navigation, advertisements, tracking scripts, cookie banners, and dynamic content that a language model has to wade through to find the actual substance. Even with context windows now measured in hundreds of thousands of tokens, a full HTML crawl of a medium-sized site exceeds what most inference workflows can fit. llms.txt solves the curation problem by letting the site owner declare what matters.

llms.txt vs robots.txt: What's the Difference?

This is the question I hear most often from clients. The two files look similar at a glance — both sit at the domain root, both are plain text, both target automated systems — but they solve opposite problems.

Both files are additive. You do not choose between them — you publish both. In fact, your robots.txt is what determines whether AI crawlers can even fetch your llms.txt file in the first place. If your robots.txt blocks GPTBot, ClaudeBot, or PerplexityBot, those systems will never read your curated content guide regardless of how well-written it is.

llms.txt File Structure and Syntax

The official specification at llmstxt.org keeps things simple. An llms.txt file must begin with an H1 declaring the project name, and everything after that is optional but recommended.

1. 1. H1 heading (required). The name of your project, business, or site. Example: # Verlua
2. 2. Blockquote summary (optional, recommended). One or two sentences describing what your site is and who it serves. Example: > Sacramento web design agency building Next.js sites for service businesses.
3. 3. Free-text context (optional). Paragraphs of prose that explain your site, its conventions, or anything an AI should know before using the linked content.
4. 4. H2 section headings (optional). Each H2 groups a set of related resources. Common sections: Services, Products, Documentation, Case Studies, About, Contact.
5. 5. Bulleted Markdown links (optional). Under each H2, list links in the form - [Link text](url): Short description of the page.

An optional convention some sites use is a final H2 called "Optional" that lists lower-priority resources an AI can skip if context is tight. Anthropic uses this pattern in its own llms.txt.

Full llms.txt Example for a Small Business

Here is a complete, realistic llms.txt for a fictional Sacramento plumbing company. Copy this structure and adapt the content to your own business. Every section below is optional except the H1, but including all of them produces the most useful file.

# Capital Plumbing Co.

> Licensed, family-owned plumbing company serving Sacramento, Elk Grove,
> Folsom, and Roseville since 2008. 24/7 emergency service, upfront
> pricing, and California Contractor License #987654.

Capital Plumbing Co. is a residential and light commercial plumbing
contractor. We handle drain cleaning, water heater replacement, leak
detection, sewer line repair, and bathroom remodels. All work is
performed by licensed plumbers and backed by a one-year labor warranty.

## Services

- [Drain Cleaning](https://capitalplumbing.example/services/drain-cleaning): Hydrojetting, snaking, and camera inspection. $149 starting price.
- [Water Heater Repair & Install](https://capitalplumbing.example/services/water-heater): Tank and tankless, gas and electric. Same-day replacement available.
- [Leak Detection](https://capitalplumbing.example/services/leak-detection): Acoustic and thermal imaging. $0 trip fee for confirmed leaks.
- [Sewer Line Repair](https://capitalplumbing.example/services/sewer-line): Trenchless pipe lining and traditional excavation.
- [Emergency Plumbing](https://capitalplumbing.example/services/emergency): 24/7 dispatch, typical on-site arrival within 60 minutes.

## Service Areas

- [Sacramento](https://capitalplumbing.example/sacramento-ca): Primary service area, full coverage including downtown and East Sac.
- [Elk Grove](https://capitalplumbing.example/elk-grove-ca): Full coverage, typical same-day response.
- [Folsom](https://capitalplumbing.example/folsom-ca): Full coverage, typical same-day response.
- [Roseville](https://capitalplumbing.example/roseville-ca): Full coverage, typical same-day response.

## Pricing & Financing

- [Upfront Pricing](https://capitalplumbing.example/pricing): Flat-rate pricing book with most common repairs listed.
- [Financing Options](https://capitalplumbing.example/financing): 0% APR financing for 12 months on qualifying repairs.

## About

- [About Us](https://capitalplumbing.example/about): Company history, licensing, and team bios.
- [Reviews](https://capitalplumbing.example/reviews): 900+ verified Google reviews, 4.9-star average.
- [Contact](https://capitalplumbing.example/contact): Phone, email, online booking, and service area map.

## Optional

- [Blog](https://capitalplumbing.example/blog): Educational articles on common plumbing issues.
- [Warranty](https://capitalplumbing.example/warranty): Full warranty terms and claim process.

This example is roughly 350 words and covers everything an AI would need to accurately describe the business, compare it to competitors, verify service areas, quote starting prices, or route a user to a contact page. A file like this takes 20 to 30 minutes to produce if the business already has its service pages and pricing published.

Where to Host llms.txt and Which MIME Type to Use

The file must live at the root of your primary domain: yoursite.com/llms.txt. Subdomains can have their own llms.txt (docs.yoursite.com/llms.txt), but the root version is what most AI systems will check first.

For hosting, the process depends on your stack:

• Next.js: Place the file at /public/llms.txt. Next.js serves the public directory at the root automatically.
• WordPress: Upload llms.txt to the root directory via FTP or cPanel file manager. Some SEO plugins now include a generator.
• Astro, Nuxt, SvelteKit: Place the file in /public/ or the equivalent static assets directory.
• Vercel, Netlify, Cloudflare Pages: These platforms serve files from your build output directory automatically — no additional config needed.
• Shopify: Shopify does not natively expose custom root files. Use a Liquid redirect or a Cloudflare Worker as a workaround.

The MIME type should be text/plain or text/markdown. Most static hosts serve .txt as text/plain by default, which is acceptable. If your host supports content negotiation, you can explicitly set text/markdown. Avoid text/html, which will cause some AI parsers to treat the content as a rendered page.

Which AI Systems Currently Read llms.txt?

Adoption is partial and changing fast. I am keeping this section narrow to what has been publicly confirmed as of early 2026.

Anthropic publishes its own llms.txt at docs.anthropic.com/llms.txt and has made the format central to how Claude retrieves context from supported documentation sites.

Perplexity references the standard in developer documentation and is widely reported to incorporate llms.txt data when available.

Google has publicly stated that its ranking systems and AI Overviews do not use llms.txt. In a statement reported by Search Engine Land, Google Search Advocate John Mueller confirmed that Google does not read the file. This does not mean Google ignores AI citations entirely — it just means the llms.txt file is not the signal.

OpenAI / ChatGPT has not officially confirmed llms.txt support for ChatGPT's web browsing or search features. Anecdotal reports suggest ChatGPT sometimes fetches the file, but there is no public specification of how it is weighted. For broader visibility in ChatGPT, our guide to getting cited in ChatGPT, Perplexity, and Gemini covers the tactics that measurably work today.

What Is llms-full.txt?

llms-full.txt is a companion file that includes the full Markdown content of the pages listed in your llms.txt, concatenated into a single document. Where llms.txt is the table of contents, llms-full.txt is the bound book. An AI that fetches llms-full.txt gets everything in one HTTP request, which reduces crawl overhead and avoids the risk that critical pages are missed due to context limits.

Decide based on site size:

• Small sites (under 30 key pages): llms.txt alone is usually enough. The AI can crawl individual linked pages cheaply.
• Medium sites (30 to 200 key pages): Publish both. llms-full.txt becomes valuable because individual crawls add up.
• Large sites (200+ key pages): Both files are essential, and you may want to split llms-full.txt into section-specific files (llms-services.txt, llms-docs.txt) referenced from the main index.

Anthropic, Cloudflare, and Mintlify all publish llms-full.txt files alongside their llms.txt. If you are already generating clean Markdown versions of your pages (common for Next.js sites with MDX or headless CMS setups), producing llms-full.txt is mostly a build-step concatenation.

Common llms.txt Mistakes to Avoid

Most llms.txt files I audit have at least one of these issues. None of them are fatal, but all of them reduce the file's usefulness.

1. 1. Broken or redirecting links. If your linked URLs 404 or redirect through multiple hops, AI systems may skip them or cache outdated versions. Audit quarterly.
2. 2. Missing the blockquote summary. The summary is the first thing an AI reads after the H1. Skipping it forces the model to infer what your site is about from link titles alone.
3. 3. Listing every page. llms.txt is curation, not a sitemap. If you list 500 pages, the most important ones get lost. Limit to 10 to 50 links for most sites.
4. 4. Forgetting the descriptions. A link without a description tells the AI nothing about what it will find at that URL. Always include a colon and a short description after the link.
5. 5. Using rendered HTML instead of Markdown. Some site generators output HTML into llms.txt by accident. This defeats the purpose of the format. Check the raw response at yoursite.com/llms.txt in a terminal with cURL.
6. 6. Outdated content. If your llms.txt lists a product you discontinued or a service area you no longer cover, AI systems may surface wrong information. Tie llms.txt updates to the same review cadence as your sitemap.
7. 7. Blocking AI crawlers in robots.txt. The most common fatal mistake. If GPTBot, ClaudeBot, and PerplexityBot cannot fetch your llms.txt, the file might as well not exist. Review your AI crawler access alongside your llms.txt work.

How to Generate llms.txt Programmatically

For sites with dozens or hundreds of pages, maintaining llms.txt by hand gets painful fast. The better approach is to generate it at build time from the same data source that powers your sitemap.

Next.js Approach

Create a route handler at app/llms.txt/route.ts that reads your service pages, blog posts, and core pages from the same source your sitemap uses, then returns a Markdown response. This lets llms.txt update automatically whenever you add new content — no manual maintenance.

// app/llms.txt/route.ts
import { services } from '@/lib/services-data';
import { blogPosts } from '@/app/blog/page';

export async function GET() {
  const base = 'https://www.yoursite.com';

  const body = `# Your Business Name

> One-sentence summary of what you do and who you serve.

## Services

${services.map(s => `- [${s.title}](${base}${s.href}): ${s.shortDescription}`).join('\n')}

## Recent Articles

${blogPosts.slice(0, 10).map(p =>
  `- [${p.title}](${base}/blog/${p.slug}): ${p.excerpt}`
).join('\n')}

## Contact

- [Contact us](${base}/contact): Primary contact form and phone.
`;

  return new Response(body, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

WordPress Approach

Several SEO plugins now include llms.txt generators. Yoast, Rank Math, and AIOSEO have all added or are testing the feature as of late 2025. For a lightweight option, you can use a custom functions.php snippet that hooks into the template_redirect action and outputs a generated Markdown file when /llms.txt is requested. The logic mirrors the Next.js example: pull your service CPT, your latest posts, and your core pages, then concatenate them into Markdown.

Third-Party Generators

If you want something faster than a build script, tools like llmstxt.firebaseapp.com and the llms-txt npm package crawl your sitemap and produce a starter file you can edit. Treat generator output as a draft, not a final deliverable — the descriptions will need human editing.

Does llms.txt Actually Improve Rankings or Citations? An Honest Answer

No controlled study has demonstrated a direct lift in AI citations or rankings from publishing an llms.txt file. Google has stated publicly that it does not use llms.txt. Other platforms have not published adoption data or weighting details.

My honest read based on deploying the file across dozens of client sites since late 2024:

• llms.txt rarely produces an overnight lift in AI citation frequency on its own.
• It does align with every other GEO best practice — clean entity definitions, clear service descriptions, linked resource hierarchy — so the process of building it tends to improve the site even when the file itself does not move the needle.
• For platforms that do read it (Anthropic, Perplexity, Mintlify-powered docs), it measurably reduces misrepresentation. I have seen Claude cite a client's stated starting price accurately instead of hallucinating a number.
• The downside risk is zero. An unused file costs nothing.
• The upside is optionality. If AI platforms expand support, sites that already have a well-curated llms.txt are ready.

Pair llms.txt with the things that actually move the needle today: structured data, topical authority, third-party citations, and content that answers the exact questions users ask AI systems. Our guide to Google AI Overviews optimization covers what works on the platform that llms.txt explicitly does not touch.

30-Minute llms.txt Implementation Checklist

If you only have half an hour, here is the minimum path to a useful llms.txt file.

• Minute 0-5: Confirm your robots.txt allows GPTBot, ClaudeBot, PerplexityBot, and Google-Extended.
• Minute 5-15: Draft your H1, blockquote summary, and a single context paragraph.
• Minute 15-25: Add H2 sections for Services, Service Areas (if local), Pricing, and About. Include 3 to 8 links per section with short descriptions.
• Minute 25-28: Save as llms.txt and place at /public/llms.txt (Next.js) or the domain root (WordPress/other).
• Minute 28-30: Fetch yoursite.com/llms.txt in a browser and confirm it returns plain text, not HTML or a 404.

Frequently Asked Questions About llms.txt