Skip to content
The Docs Report
Tuesday, November 17, 2026NewsletterRSS

ReadMe Sends AI Agents Back to llms.txt When They Guess a Wrong URL

ReadMe's Oct. 7 update nests llms.txt under sidebar categories and redirects guessed or missing markdown URLs to the index.

2 min read

ReadMellms.txt

Product photo of a white document folder on a hot pink, magenta and orange gradient

ReadMe now redirects AI agents that request a missing or guessed markdown URL to the project’s llms.txt file, ReadMe said in its Oct. 7 changelog. Requests for a missing .md page return a markdown 404 that points to the index. Guessed paths such as /docs.md, /reference.md and /docs/<category>.md redirect to the matching llms.txt. Markdown requests to a project root redirect there too.

llms.txt now follows the sidebar

The llms.txt file groups pages under their sidebar categories, with child pages nested under their parents. API Reference intro pages sit under a plain API Reference heading. ReadMe replaced its llms.txt settings with a single Summarize Index option. It links the root file to per-section and per-category files instead of listing every page.

Markdown pages point agents to the index

Markdown versions of ReadMe pages now carry agentTools front matter with links to the llms.txt index and a search endpoint. The markdown versions of the API Reference Getting Started and Authentication pages now include the API’s servers and security schemes when the OpenAPI definition is visible to the reader.

Other changes in the release

ReadMe said hidden and deprecated endpoints can now be turned on one at a time for a project’s MCP server from the MCP routes settings. Each AI Agent edit suggestion, whether a page rewrite, an API Reference update or a Recipe change, has its own Apply and Revert controls. Enterprise group analytics can be exported to CSV. Translated pages can be re-translated one at a time.

Key takeaways

  • Missing or guessed markdown URLs now redirect or 404 back to llms.txt.
  • The llms.txt file mirrors sidebar categories, with a Summarize Index option that links to per-section files.
  • Hidden and deprecated endpoints can be enabled individually for MCP.

The take

We think the redirect is the most useful change. An agent that has read one page can try /reference.md or a category path next. A dead end there leaves it with a search result or its training data. A redirect hands it the index.

The Summarize Index option has a cost. A root file that only links to section files makes an agent fetch twice before it reads a page. A large project can absorb that. A small one gains nothing from it.

We’d test a docs site the way an agent would. Request a few made-up .md paths and check where each one lands. Then see whether llms.txt still lists every page an agent should find.