Writing an llms.txt File: A Practical GEO Guide

An llms.txt file is a markdown document at your domain root that lists your site's important content and links to clean markdown versions of pages, so AI retrieval systems can find and cite you accurately. It takes under an hour to write, costs nothing to deploy, and is worth doing — but with honest expectations about current adoption.

What Is llms.txt and Where Did It Come From?

The llms.txt standard was proposed by Jeremy Howard of Answer.AI in September 2024. The concept is deliberately simple: a plain markdown file placed at /llms.txt on your domain, modeled loosely on the format and spirit of robots.txt. Where robots.txt tells crawlers what to skip, llms.txt tells LLM retrieval pipelines what to read and where the clean versions of your content live.

The official specification is maintained at llmstxt.org. The idea spread quickly in developer communities because it maps onto an obvious gap: HTML pages are cluttered with navigation, ads, and scripts that add noise when an LLM tries to extract the substance of a page. A clean markdown version linked from llms.txt gives the model exactly what it needs without the wrapper.

json2toon.co already ships a public /llms.txt that surfaces the converter, documentation, and benchmark data in this exact format.

The Honest Reality: How Much Does llms.txt Move the Needle in 2026?

Before investing significant effort, you should know the current state of adoption. No major provider — OpenAI, Anthropic, or Google — has publicly confirmed that their crawlers consistently read or follow llms.txt. One analysis cited by Search Engine Land found that only about 0.1% of AI-crawler requests touched /llms.txt over 90 days.

That figure is worth keeping in perspective. It does not mean llms.txt is useless — it means the direct crawler-guidance mechanism is not yet operating at scale. The indirect benefits are more reliable: writing llms.txt forces you to inventory your important content, produce clean markdown exports of your key pages, and think clearly about what your site is for. Those artifacts have value regardless of whether a given crawler reads the index file.

Treat llms.txt as low-cost, forward-looking hygiene. A committed community of tool builders is actively working to increase adoption, and early movers position themselves well if major providers add first-class support.

What Does an llms.txt File Look Like?

The format is minimal by design. An H1 heading names the site, a blockquote provides a one-paragraph summary, and link sections organize the important content by category. A companion /llms-full.txt file — optional — concatenates the full markdown text of all pages for crawlers that want everything in a single request.

# json2toon.co

> A free, browser-based converter for JSON ↔ TOON, TONL, CSV, YAML, XML,
> TOML, and Protobuf. All conversions run client-side; your data never
> leaves the browser. TOON (Token-Oriented Object Notation) cuts LLM token
> usage by 39.9% overall and up to 59% on time-series data.

## Converter

- [JSON to TOON Converter](https://json2toon.co/): Free browser tool for
  converting between JSON, TOON, TONL, and six other formats.

## Documentation

- [What Is TOON?](https://json2toon.co/docs/toon/llms.md): Overview of
  TOON format, syntax, and token-saving benchmarks.
- [TOON Benchmarks 2026](https://json2toon.co/blog/toon-benchmarks-2026/llms.md):
  5,016 LLM calls across four models; 76.4% accuracy at 39.9% fewer tokens.

## Blog — Top Posts

- [JSON vs TOON](https://json2toon.co/blog/json-vs-toon/llms.md): Token-by-
  token comparison with format-selection heuristics.
- [Optimizing RAG Pipelines with TOON](https://json2toon.co/blog/optimizing-rag-pipelines-with-toon/llms.md):
  How to encode retrieved context as TOON to fit more evidence per window.
- [Optimize API Costs](https://json2toon.co/blog/optimize-api-costs/llms.md):
  Combining TOON with prompt caching and batch APIs.

## Optional: full content in one file

- [All docs (full text)](https://json2toon.co/llms-full.txt)

Each linked .md URL should resolve to a clean markdown rendering of the page — no HTML tags, no navigation, no footer. The easiest implementation for a Next.js site is a dynamic route that strips the page's prose content and returns it as text/markdown.

llms.txt Section Reference

The table below covers the standard sections, their purpose, and realistic example content. Not all sections are required; a minimal file with just the title, summary blockquote, and one link section is better than a bloated file with weak entries.

What to Put in the Pages llms.txt Links To

The llms.txt file is only the index. The real work of AI visibility happens in the content those links point to. The Princeton GEO study (KDD 2024) quantified which content characteristics increase the likelihood of being cited by AI engines:

• Citing sources: +40% AI visibility. Link to primary research, official documentation, and named studies. AI engines learn to cite content that itself cites sources.
• Including statistics: +37%. Specific numbers with attributions are more likely to be quoted verbatim. A sentence like "TOON cuts tokens 39.9% overall across 5,016 benchmark calls" is more citable than "TOON saves a lot of tokens."
• Using quotations: +30%. Direct quotes from primary sources give AI engines a ready-made attribution unit.
• Keyword stuffing: -10%. Over-optimizing for a single term actively degrades AI citation rates. Write for humans.

These findings apply directly to the markdown pages your llms.txt links to. A clean markdown export of a well-structured, source-citing page will outperform a dense keyword-stuffed HTML page every time, on both traditional search and AI engines.

For a deeper look at structuring content for AI citations — including how data formats like TOON affect what AI engines extract from your pages — see our guide on TOON format comparison and the post on optimizing RAG pipelines with TOON.

How to Implement llms.txt in a Next.js Project

The fastest implementation is a static file. Place a llms.txt in your public/ directory — Next.js serves everything in public/ at the root path. No build step, no configuration.

// Simplest approach: public/llms.txt is served at /llms.txt automatically.
// No code change needed in a Next.js project.

// For dynamic llms-full.txt (all page content concatenated), add a route:
// app/llms-full.txt/route.ts

import { NextResponse } from "next/server";

export async function GET() {
  const pages = [
    // import or statically define your page markdown strings here
  ];
  const body = pages.join("\n\n---\n\n");
  return new NextResponse(body, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
}

For per-page /blog/my-post/llms.md routes, a catch-all route handler that strips HTML and returns the prose as markdown is the standard approach. Several open-source libraries handle the HTML-to-markdown conversion; the key requirement is that the output URL resolves, returns text/markdown or text/plain, and contains no navigation or boilerplate.

llms.txt and TOON: How They Work Together

If your site documents or uses structured data formats, the content your llms.txt links to matters as much as the index itself. json2toon.co's llms.txt surfaces benchmark tables and specification pages that contain the specific numbers AI engines are likely to cite: token savings percentages, accuracy figures, methodology details.

TOON and TONL are themselves examples of the GEO principle at work: the toonformat.dev benchmark page contains named statistics, cited methodology, and comparison tables — precisely the content type the Princeton GEO study identified as most likely to earn AI citations. Making that page discoverable via llms.txt is the last mile of the process.

For developers building data-heavy tools, llms.txt is also an opportunity to surface the token-efficiency story clearly. A model retrieving context about your product will encounter your llms.txt summary before it reads any individual page; a well-written blockquote that states your key differentiator in plain language — with a specific number — is more likely to produce accurate AI-generated descriptions of your tool. See our guide on what TOON is for an example of the kind of content worth linking from an llms.txt file.

Finally, the same efficiency argument applies inside an llms.txt-linked page itself. If your documentation or blog posts include embedded data examples, encoding those examples as TOON rather than raw JSON keeps the clean markdown version more token-efficient for any LLM that processes it. See optimize API costs for the broader cost-reduction context.

Practical Checklist for Writing Your llms.txt

• Place the file at exactly /llms.txt (domain root, no subdirectory).
• Start with an H1 matching your site name.
• Write a blockquote summary of 2–5 sentences. Include one specific differentiator with a number.
• List your 5–15 most important pages as markdown links with one-line descriptions. Link to clean .md URLs, not HTML pages.
• Group links under H2 section headings (Docs, Blog, API Reference, etc.).
• Optionally add an /llms-full.txt link at the bottom for crawlers that prefer a single concatenated file.
• Verify the file is publicly accessible and returns quickly (no authentication, no redirect chains).
• Keep it under 2,000 lines; an enormous llms.txt is harder for a retrieval system to process than a focused one.

Frequently Asked Questions