Updating…

🍪We use cookies!

This website utilizes cookies to enable essential site functionality, as well as for analytics, personalization, and targeted advertising. You may change your settings at any time or accept the default settings.

CaptivateClick
  • Home/
  • Services/
    Web Agency
    • Web designResponsive sites built to convert.
    • Branding & identityBrand identity, logo and visual system designed to stay distinctive and consistent as you grow.
    • E-commerceWebshops built to sell.
    • Shopify agencyShopify stores that sell.
    • Landing pagesCampaign and offer pages.
    • PricingWhat a new business website costs.
    Lead Generation
    • Cold emailRelevant outreach that starts conversations.
    • LinkedIn outreachSocial selling with a human voice.
    • Appointment settingQualified B2B meetings in your calendar.
    • ProspectingThe right companies, people and timing.
    SEO
    • Technical SEOHelp Google discover, read and index the right pages.
    • Local SEOMaps and local visibility.
    • SEO auditSee what holds you back and what to fix first.
    • Get found on GoogleHelp customers find your business.
    Advertising
    • Ad reviewLanding page for your ad sends.
    • Google AdsGoogle Ads and SEM built around high-intent searches, qualified conversions and measurable revenue.
    • Google Ads consultantAn audit of your Google Ads account, the fixes that matter and monthly advice for your team.
    • Meta AdsFacebook and Instagram campaigns built around creative testing, clean signals and profitable scale.
    • LinkedIn AdsB2B campaigns built around account targeting, buying committees and qualified pipeline.
    AI Automation
    • AI agentsAgents that do the work.
    • Xero & QuickBooks integrationXero and QuickBooks connected to your systems.
    • HubSpot automationsHubSpot pipeline automation.
    • System integrationSystems that share the same data.
    AI visibility
    • AI checkFree AI visibility check.
    • Answer engine optimisationBecome the answer AI cites.
    • GEO SEOShow up in AI-generated answers.
    • AI search (guide)Guide: what AI search is and what it means.
    See all services
  • Projects/
    • Climentum CapitalWeb designView case
    • Nordic Acceleration GroupWeb designView case
    • NorthforkWeb designView case
    • Hawkeye AdvertisingWeb designView case
    All projects
  • About Us/
  • Blog/What is llms.txt, and does your website need one?
  • Contact/

Blog

What is llms.txt, and does your website need one?

November 17, 20269 min read
A person types on a white keyboard in front of a monitor showing a dark code editor on a light wooden desk.

On this page

  • What is llms.txt?
  • What does an llms.txt file look like?
  • llms.txt vs robots.txt vs sitemap.xml: what is the difference?
  • Do Google, ChatGPT and other AI tools use llms.txt?
  • Google Search, AI Overviews and AI Mode
  • Chrome Lighthouse
  • ChatGPT, Claude and Perplexity
  • Coding agents and documentation tools
  • How do you create an llms.txt file?
  • Using an llms.txt generator
  • Is llms.txt worth adding to a business website?
  • What matters more for AI visibility than llms.txt?
  • Frequently asked questions
  • Sources

Short answer: llms.txt is a Markdown file, usually placed at /llms.txt, that gives AI agents a short, curated overview of a website and links to its most important pages. Jeremy Howard proposed it in September 2024 and revised it in August 2026. It has not been adopted as an official web standard. Google says Google Search ignores the file, and the crawler documentation from OpenAI, Anthropic and Perplexity does not name it as something that affects their answers. It is cheap to add and most useful where coding agents read documentation. For an ordinary business website it is optional.

In brief

An llms.txt file needs only one thing: an H1 with the name of the site. A summary and lists of links are optional. It does not control crawling (robots.txt does) and it does not replace a sitemap. We found no first-party source saying that any AI assistant uses the file to decide which businesses to cite, and Google states that the file neither helps nor harms visibility in Search. Add one if your CMS generates it or if you publish documentation. Put crawl access, clear content and mentions on other sites first.

Contents

  1. What is llms.txt?
  2. What does an llms.txt file look like?
  3. llms.txt vs robots.txt vs sitemap.xml: what is the difference?
  4. Do Google, ChatGPT and other AI tools use llms.txt?
  5. How do you create an llms.txt file?
  6. Is llms.txt worth adding to a business website?
  7. What matters more for AI visibility than llms.txt?

What is llms.txt?

llms.txt is a plain text file written in Markdown that tells an AI agent what a website is about and where its most useful pages are.

The proposal comes from Jeremy Howard, who published it on 3 September 2024. A second version followed in August 2026 (llmstxt.org). The page still describes itself as a proposal. No standards body has adopted it, and nothing obliges an AI service to read it.

The reasoning is practical. Web pages are built for people, with the information wrapped in navigation, ads and JavaScript, and an agent's context window is too small for most whole websites. A short file that points to the pages that matter saves the agent the guessing.

According to the proposal, agents view or search the file and then follow the relevant links. It is used most heavily for software documentation, and the same structure can work for a business outlining its structure and policies.

What does an llms.txt file look like?

An llms.txt file is a short Markdown document with a fixed order of parts. The specification lists them like this:

  1. An H1 with the name of the project or site. This is the only required part.
  2. A blockquote with a short summary containing what a reader needs in order to understand the rest of the file.
  3. Free text, such as paragraphs or lists, with more detail. Any Markdown is allowed here except headings.
  4. H2 sections with link lists. Each item is a Markdown link, optionally followed by a colon and a note about the page.

Here is an llms.txt example for a fictional heating company:

# Example Heating Ltd

> Example Heating Ltd installs and services heat pumps and boilers for homes in the Northfield area. Quotes are fixed after a home survey.

Pages marked "guide" give general advice and do not replace a survey.

## Services

- [Heat pump installation](https://www.example.com/services/heat-pumps.md): What is included, the usual timeline and how quotes work
- [Boiler servicing](https://www.example.com/services/boiler-service.md): What an annual service covers

## Company

- [About us](https://www.example.com/about.md): Who we are, certifications and service area
- [Contact](https://www.example.com/contact.md): Phone, email and opening hours

## Optional

- [Guide: choosing a heat pump](https://www.example.com/guides/choosing-a-heat-pump.md)
  • The "Optional" heading has a meaning. By convention it holds secondary links that an agent can skip when it needs a shorter context.
  • The links should lead to text an agent can read easily. The proposal recommends offering a clean Markdown version of important pages at the same address with .md added, which is why the example links end in .md. If your site cannot serve Markdown versions, link to the normal pages.
  • The file can sit in a subfolder. A file at /docs/llms.txt covers the pages under /docs/.

llms.txt vs robots.txt vs sitemap.xml: what is the difference?

The three files answer different questions. robots.txt says what automated tools may access, sitemap.xml lists the pages a search engine can index, and llms.txt is a curated summary with links for an agent that is looking something up. None of them replaces another.

robots.txtsitemap.xmlllms.txt
PurposeStates which crawlers may access which parts of the siteLists the indexable pages for search enginesGives agents a short overview and links to key pages
FormatPlain text rulesXMLMarkdown
StatusEstablished and documented by Google, OpenAI, Anthropic and PerplexityEstablished and used by search enginesA proposal

The proposal draws the same line: robots.txt tells automated tools what access is acceptable, while llms.txt is read on demand when an agent needs information while assisting a user. A sitemap is no substitute, because it usually lists far more than fits in a context window.

One practical consequence: you cannot use llms.txt to block or allow an AI crawler. That is still done in robots.txt.

Do Google, ChatGPT and other AI tools use llms.txt?

Google Search does not, by its own account. For ChatGPT, Claude and Perplexity we found no first-party statement that the assistants use a site's llms.txt when they answer. The documented users are mainly coding agents and documentation tools. We checked the sources below on 17 November 2026.

Google Search, AI Overviews and AI Mode

Google's guidance is explicit. Its guide to generative AI features in Search says you do not need to create new machine readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI capabilities, because Google Search does not use them. It adds that it is fine to maintain llms.txt files for other services that use them, and that doing so will neither harm nor help visibility or rankings in Google Search (Google Search Central).

The page on AI features says the same from the other direction: there are no additional requirements for appearing in AI Overviews or AI Mode. To be shown as a supporting link, a page must be indexed and eligible to appear in Search with a snippet (Google Search Central).

Chrome Lighthouse

Another Google product does look at the file. Lighthouse, the audit tool in Chrome, has an Agentic Browsing category that checks whether a machine-readable summary exists at the domain root. The Chrome documentation calls the category experimental and based on proposed standards, and it does not connect the check to Google Search (Chrome for Developers). A pass there tells you the file exists and nothing about rankings.

ChatGPT, Claude and Perplexity

OpenAI, Anthropic and Perplexity each document the bots they run and how to allow or block them in robots.txt. OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers (OpenAI). Anthropic says its bots honour robots.txt and that blocking Claude-SearchBot or Claude-User may reduce a site's visibility (Anthropic). Perplexity recommends allowing PerplexityBot in robots.txt (Perplexity).

None of these pages says the company's assistant fetches a website's llms.txt to answer a question, or that having one changes whether a site is cited. Anthropic's page does not mention the file. OpenAI's and Perplexity's pages mention it only as a link to the index of their own documentation.

OpenAI, Anthropic, Perplexity and Google's Gemini API all publish an llms.txt for their developer documentation. That shows the labs find the format useful for their own docs. It does not show that their chat products look for yours.

Coding agents and documentation tools

This is where the proposal reports real use. Its author writes that coding agents use the files reliably and that documentation platforms generate one automatically (llmstxt.org, changes in v2).

The summary: there is no evidence from the companies behind the big assistants that llms.txt gets a business mentioned or cited more often, and Google says outright that it has no effect in Search.

How do you create an llms.txt file?

You create an llms.txt file by writing a short Markdown document by hand or letting your CMS generate one, and publishing it at the root of your domain.

  1. Pick the pages. Choose the few pages a stranger would need in order to understand what you sell, to whom, where and on what terms. Leave out the rest.
  2. Write the H1 and the summary. Use your business name as the H1. In the blockquote, say in one or two sentences what you do and for whom.
  3. Group the links under H2 headings. Add a short note after each link that says what the page answers. Put nice-to-have material under "Optional".
  4. Save it as llms.txt and upload it to the root so it opens at yourdomain.com/llms.txt. Check in a browser that you get the text file and not an error page or your home page.
  5. Test it. The proposal suggests giving an agent only your llms.txt as a starting point and asking it questions about your content.
  6. Keep it current. A file that points to removed pages or old prices is worse than no file.

Using an llms.txt generator

Many platforms now do the work for you. The proposal's own list of integrations includes the WordPress plugins Yoast SEO and AIOSEO, Wix, which generates the file for every Wix site, and the documentation platforms Mintlify and GitBook. Check your plugin or platform settings before writing anything by hand.

Whatever generator you use, read the result. A file that lists every URL on the site repeats the sitemap and loses the benefit of a curated overview.

Is llms.txt worth adding to a business website?

For most business websites llms.txt is worth adding only if it costs you almost nothing. It will not hurt, and it has no documented effect on being recommended by AI assistants.

It makes sense when:

  • your CMS or SEO plugin generates the file and you only need to review it
  • you publish technical documentation, an API or detailed product manuals that coding agents and other tools are likely to read.

It is not worth prioritising when:

  • you would pay for it as a separate project while robots.txt blocks AI crawlers or key pages are not indexed
  • someone promises that the file will get you into ChatGPT or Google's AI answers. No source we could open supports that promise.

Our position is the same as in the article GEO vs SEO: treat it as optional housekeeping.

What matters more for AI visibility than llms.txt?

The things with documented effect are crawl access, indexable pages and content that answers real questions. Start there.

  1. Let the search bots in. Check robots.txt for OAI-SearchBot, PerplexityBot, Claude-SearchBot and Googlebot. OpenAI says a robots.txt change takes about 24 hours to reach its search systems.
  2. Make sure important pages are indexed. Google requires a page to be indexed and eligible for a snippet before it can be a supporting link in AI Overviews or AI Mode.
  3. Say plainly what you offer. Each service page should state what you do, for whom and where.
  4. Keep your facts consistent. Your name, services and contact details should match on your own site, in directories and in reviews.
  5. Measure. Ask the main assistants the questions your customers ask and note whether you are mentioned. Our AI check gives a quick first snapshot.

How we do this as ongoing work is described on our page about generative engine optimization.

Does AI mention your business when customers ask?

We test the questions your customers ask in several AI assistants and show which companies are mentioned, which sources are cited and what to fix first. A first call is free and non-binding.

See how we work with AI visibility
CaptivateClick

Navigation

  • Home
  • Services
  • Projects
  • About Us
  • Blog
  • Contact
  • Privacy Policy

Services

  • Website Design
  • Lead Generation
  • SEO
  • Advertising
  • AI Automation
  • AI visibility

Web Agency

  • Web design
  • Branding & identity
  • E-commerce
  • Shopify agency
  • Landing pages
  • Pricing

Lead Generation

  • Cold email
  • LinkedIn outreach
  • Appointment setting
  • Prospecting

SEO

  • Technical SEO
  • Local SEO
  • SEO audit
  • Get found on Google

Advertising

  • Ad review
  • Google Ads
  • Google Ads consultant
  • Meta Ads
  • LinkedIn Ads

AI Automation

  • AI agents
  • Xero & QuickBooks integration
  • HubSpot automations
  • System integration

AI visibility

  • AI check
  • Answer engine optimisation
  • GEO SEO
  • AI search (guide)
© 2026 CaptivateClick

Frequently asked questions

What is llms.txt in simple terms?

It is a small Markdown file, usually at /llms.txt, that summarises a website for AI agents and links to its most important pages. Jeremy Howard proposed it in September 2024 and updated the proposal in August 2026. It is a proposal and has not been adopted as an official web standard.

Does Google use llms.txt?

Not in Google Search. Google's guidance says Search does not use such files and that maintaining one will neither harm nor help visibility or rankings. Chrome's Lighthouse tool has an experimental check for the file in its Agentic Browsing category, which is separate from Search.

Does ChatGPT read llms.txt?

OpenAI's crawler documentation does not say so. It describes OAI-SearchBot, GPTBot and ChatGPT-User and explains how robots.txt controls them. We found no first-party statement that ChatGPT uses a website's llms.txt when it answers or chooses sources.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may access. llms.txt is a curated overview that an agent can read when it needs information, and it cannot allow or block anything. If you want to control AI crawlers, do it in robots.txt.

Where should the llms.txt file be placed?

Normally at the root of the domain so that it opens at yourdomain.com/llms.txt. The proposal also allows a file in a subfolder, for example /docs/llms.txt, which then covers the pages under that path.

Is there an llms.txt generator?

Yes. According to the proposal's list of integrations, Yoast SEO and AIOSEO generate the file in WordPress, Wix creates it for every Wix site, and documentation platforms such as Mintlify and GitBook do the same. Review the generated file so that it stays a short, accurate selection.

Sources

  1. The /llms.txt file, v2, llmstxt.org (Jeremy Howard)
  2. Changes: v2 (August 2026), llmstxt.org (Jeremy Howard)
  3. Optimizing your website for generative AI features on Google Search, Google Search Central
  4. AI features and your website, Google Search Central
  5. Lighthouse agentic browsing scoring, Chrome for Developers
  6. Overview of OpenAI Crawlers, OpenAI
  7. Does Anthropic crawl data from the web, and how can site owners block the crawler?, Anthropic
  8. Perplexity Crawlers, Perplexity
Let's get startedLet's get started

Ready to grow?

Let's talk about your project.

Trusted by 110+ businesses to grow online.

  • Mercer Consulting, Co.
  • Mercer Consulting, Co.
PreviousGEO vs SEO: what's the difference?

Related Posts

  • Read the blog

    GEO vs SEO: what's the difference?

    November 10, 20268 min read
    Read more
  • Read the blog

    How to appear in ChatGPT: 7 reasons you are not mentioned, and what to do

    October 8, 20265 min read
    Read more
  • Read the blog

    How AI chooses which companies to recommend

    October 8, 20265 min read
    Read more