# llms.txt v2: what the new version changes, and what we did with it on our own site > The llms.txt proposal moved to version 2 on 10 August 2026: standard links, Markdown versions of pages, per-subdirectory files. What it changes, what it does not do, and what we measured when applying it on dotsland.com. - Original page: https://dotsland.com/en/llms-txt-v2-what-changes/ - Language: en - Published: 2026-09-25T09:00:00+02:00 - Updated: 2026-09-24T20:50:28+02:00 - Section: Digital Marketing - Publisher: Dotsland, https://dotsland.com/en/ --- ## The short version - **The llms.txt proposal moved to version 2 on 10 August 2026.** It adds three things: standard links so an agent can find the file and the Markdown version of a page, Markdown versions of the pages themselves, and files per subdirectory. - **It is still a proposal, not a standard.** The Chrome audit that checks it only flags a server error; a missing file is marked not applicable, because providing it remains optional. - **We applied it on dotsland.com**, and we publish here what we measured: our articles and expertise pages now exist in Markdown, three to five times lighter than their HTML page. We also publish what our server logs show: for now, very few agents come to read these files. - **The most misunderstood rule is the subdirectory one**: a file only covers the pages under its path. Copying another site’s structure often means publishing files that cover nothing. Run the test. Type your site’s address followed by `/llms.txt`. Then take one of your articles and add `index.md` to the end of its address. On our site, the first returns an index of 72 links, and the second returns the text of the article, with no menu, no footer, no script. If yours returns a 404 twice, this article tells you what version 2 changes, and what it does not. ## What llms.txt is, in three ideas **A Markdown index, for agents.** The proposal, published by Jeremy Howard in September 2024, consists of placing at the root of a site a `/llms.txt` file that presents the site in a few lines and points to its useful content. The format is simple and fixed: a title, a summary, sections, and links with a short description. The starting idea has not changed: a web page is made for a human, wrapped in navigation and code, and an agent wastes time and context cleaning it up. In practice, an llms.txt file looks like this: ``` # Example Workshop > Solid wood furniture. Buying and care guides. ## Guides - [Choosing a wood species](https://example.com/guides/species/index.md): oak, walnut, ash, what changes in daily use - [Caring for oiled furniture](https://example.com/guides/care/index.md): frequency, products, common mistakes ``` A title, a one-line summary, then links grouped by section, each with a few words saying what it contains. The agent reads these few lines and picks the page to open. **It has spread through technical documentation.** Version 2 opens with that observation, which is its author’s: thousands of sites publish a file, documentation platforms generate one automatically, and the major AI labs publish one for their own developer documentation. We cannot verify the number of sites. We checked that last point: OpenAI, Anthropic and Google each serve one for their documentation. **Version 2 starts from real usage.** Where the first version, in 2024, predicted that agents would read websites, the second describes what they do: look at the index, then follow the useful links. Hence its new features, all aimed at making that path more direct. ## Where it comes from, in four dates - **3 September 2024.** Jeremy Howard publishes the proposal on llmstxt.org. - **The following two years.** According to version 2, the file settles into technical documentation: documentation platforms generate it automatically, and the major AI labs publish one for their developer documentation. - **10 August 2026.** Version 2 is released, rewritten from those two years of use. - **22 August 2026.** We publish our [citability audit](https://dotsland.com/en/ai-citability-audit/), where we write that no public evidence shows the file gets a site cited. That is still true. - **24 September 2026.** We apply version 2 on dotsland.com, a month and a half after its release. Two years between the proposal and its first overhaul, then a few weeks for a site like ours to fall behind. A file set up once and forgotten ages quickly: the question is not only whether you have one, but which version it follows. ## What version 2 changes *One surprising point: the “llms-full.txt” file, often presented as an essential companion, appears nowhere in version 2. It pushes the other way: a short index, with the detail behind the links.* | Question | Version 1 (2024) | Version 2 (10 August 2026) | | --- | --- | --- | | **How does an agent find the file or the Markdown version of a page?** | It applies an address rule, but nothing in the page signals the Markdown version | Through standard link relations: `describedby` to the llms.txt that covers the page, `alternate` with type `text/markdown` to its Markdown version. As a tag in the page, or as an HTTP header | | **Where is the Markdown version?** | `page.html.md`, and `index.html.md` for an address with no file name | A second accepted form, `page.md`, and `index.md` alongside `index.html.md` | | **What does an llms.txt in a subdirectory mean?** | Allowed, with no rule | It covers the pages under its path, and the most specific file applies | | **And the “Optional” section?** | Told tools what they could leave out | Still allowed, but with no mechanical effect: the context-expansion tool is no longer part of the proposal | ### Why a Markdown version of each page It is the most concrete new feature, and the least explained. An agent does not read a page the way you do: it fetches its entire code, menu, footer, scripts and consent banner included, then has to extract the text. The proposal puts it this way: an HTML page “wraps its information in navigation, ads, and JavaScript, and converting it back into clean text is difficult and imprecise”. The Markdown version is the text alone, with its structure: headings, lists, tables, links. - **Less noise.** The agent reads what you wrote, not your menu, and is less likely to mix your point with the rest of the page. - **Less weight.** An agent works within a limited reading window, and every useless element costs it time and money. On our site, a page weighs three to five times less in Markdown than in HTML. - **More pages read for the same effort.** An agent that consults several pages to answer a question reads more of them when they are light. The limit is the same as for the file: it helps an agent already visiting you, it does not make it come. ## The blind spot: what the file does not do Version 2 states that Chrome’s audit tool, Lighthouse, checks for an llms.txt as part of its agentic browsing audits. That is accurate, and it is often quoted as proof that the file has become essential. Lighthouse’s own documentation says something else when you read it in full: the audit only flags a **server error** when retrieving the file. If the file does not exist, the audit is marked **not applicable**, “as providing the file is optional at the moment”. Lighthouse itself describes it as an emerging convention. In other words, nothing has changed since what we wrote in August about [being cited by AI](https://dotsland.com/en/ai-citability-audit/): no public evidence shows that the file triggers citations. What version 2 improves is the experience of an agent that is already looking for something on your site. Not the likelihood that it comes. We measured it on our own site. From 31 August to 24 September 2026, our server logs show **four reads** of our llms.txt by an AI crawler whose IP address we verified (the name a crawler gives itself can be faked, its address much less so): three by OpenAI’s search crawler, one by GPTBot. None by a crawler presenting itself as Anthropic’s or Perplexity’s. Most other reads came from our own checking tools. As for our Markdown versions, put online on 24 September, no agent had read them yet that day. We will measure again in a month. > **The mistake we can see coming** > Copying another site’s structure. The subdirectory rule says a file covers the pages under its path. A site whose articles live under `/2026/` benefits from a `/2026/llms.txt`. On a site whose articles sit at the root of the address, like ours, a `/blog/llms.txt` would cover no article at all. Before creating a file per directory, look at where your addresses actually live. ## The method, and what we did on our own site We applied version 2 on dotsland.com in September 2026. The figures below were taken on 24 September, on the live site. 1. **Keep the index short.** Our French `/llms.txt` weighs 28 KB for 72 links, the English one 21 KB. That is the order of magnitude version 2 recommends: small enough to fit in an agent’s context, with the detail living behind the links. 69 of our 72 links now point to the Markdown version of the page rather than its HTML. 2. **Publish the Markdown version of the pages that matter.** Our articles and expertise pages respond at `index.md`, as `text/markdown`. The article on product order in categories weighs 87 KB in HTML and 24 KB in Markdown, 3.6 times less; the e-commerce expertise page goes from 64 KB to 12 KB, 5.1 times less. These versions are served with a `noindex` header, which asks search engines not to index them alongside the HTML page. 3. **Set both link relations**, as an HTTP header and in the page. Each article declares its Markdown version and the llms.txt that covers it. The most-specific-file rule is respected: an expertise page points to `/expertise/llms.txt`, an English article to `/en/llms.txt`. 4. **Create subdirectory files only where addresses have subdirectories.** On our site, only the expertise pages sit under a path, `/expertise/` and `/en/expertise/`: those are the only two subdirectory files we created. A five-minute self-diagnosis: open `/llms.txt` on your site, count the links and check that none returns an error. Then look at the address of three of your most important pages: do they sit under a shared directory, or at the root? The answer tells you whether subdirectory files make sense for you. Finally, if you have access to your server logs, look at who reads your llms.txt: it is the only way to know whether it is used, and by whom. ## The options, and what they are worth *These approaches combine over time: the index alone is a good first step, the full version is worth it when your content is actually consulted.* | Approach | What you get | Who it suits | | --- | --- | --- | | **Publish nothing** | No negative signal in Chrome’s audit, which marks the absence as not applicable. An agent visiting you has to clean up your HTML itself | Sites whose content agents do not consult | | **An llms.txt index alone, version 1 style** | Little work, a clean entry point. The links point to heavy HTML pages | Any site that wants a minimum without a technical project | | **Full version 2** | Index, Markdown versions, link relations, useful subdirectories. Needs development or an extension that generates the Markdown on every publication | Sites rich in reference content, documentation, product pages, guides | | **One big file with all the content** | Handy to produce, but against the direction of version 2, which does not mention this format | To avoid as the centrepiece | ## Tooling If you are looking for an extension or a development for your CMS, four criteria are enough: it generates the Markdown version from published content, with no manual entry; it sets the link relations in the HTTP header and in the page; it points to the most specific file for each address; and it keeps the index up to date on every publication, without anyone having to think about it. Method comes before the tool: a generator that creates one file per directory without looking at where your addresses live produces exactly the meaningless files the subdirectory rule helps you avoid. ## The bigger picture Version 2 records a real change: agents read websites every day, and a site can make their job easier or harder. But it remains a proposal, and the Chrome tool that checks it treats it as optional. Both are true at once, which is why the subject is so poorly told: it is presented either as a gimmick or as an obligation, when it is a technical courtesy that has become well equipped. Our reading, in our [artificial intelligence expertise](https://dotsland.com/en/expertise/artificial-intelligence/): do it, because it costs little and it genuinely serves the agents already reading you. Do not wait on it to be cited: citation happens elsewhere, in what others write about you. ## Sources The verifiable claims in this article link to their primary source, consulted on 24 September 2026. The measurements on dotsland.com were taken the same day. We do not cite a source we have not read. 1. Jeremy Howard, “The /llms.txt file, v2”, llmstxt.org, published 3 September 2024, modified 10 August 2026. [View](https://llmstxt.org/) 2. llmstxt.org, “Changes”, details of the version 2 changes. [View](https://llmstxt.org/changes.html) 3. Chrome for Developers, Lighthouse, “llms.txt” audit in the agentic browsing audits. [View](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt) 4. llms.txt files of the developer documentation of OpenAI, Anthropic and Google. [OpenAI](https://developers.openai.com/llms.txt), [Anthropic](https://docs.anthropic.com/llms.txt), [Google](https://ai.google.dev/gemini-api/docs/llms.txt) ## FAQ ### Do I also need an llms-full.txt file? Version 2 does not mention it. It recommends the opposite: a short index, with the detailed content in Markdown versions of the pages that the agent follows as needed. One big file can exist as a complement, but it does not replace that structure. ### Do the Markdown versions create duplicate content? That is the risk to handle. We serve ours with a `noindex` header, which asks search engines not to index them, and each HTML page remains the reference version. Do not publish Markdown versions without that precaution. ### Does it help my site get cited by AI? No public evidence shows it, and Chrome’s audit treats the file as optional. What it improves is how an agent already visiting your site reads it. Citation depends mostly on what other sources say about you, as we explained in our [citability audit](https://dotsland.com/en/ai-citability-audit/). ### Can Dotsland help? Yes. A diagnosis of what your site serves agents today, a choice between the index alone and the full version depending on your content, then implementation and checks. It is part of our [artificial intelligence expertise](https://dotsland.com/en/expertise/artificial-intelligence/). [Let’s talk](https://dotsland.com/en/contact-us/), or start by typing `/llms.txt` after your site’s address.