ReadMe Sends AI Agents Back to llms.txt When They Guess a Wrong URL
ReadMe's Oct. 7 update nests llms.txt under sidebar categories and redirects guessed or missing markdown URLs to the index.

ReadMe now redirects AI agents that request a missing or guessed markdown URL to the project’s llms.txt file, ReadMe said in its Oct. 7 changelog. Requests for a missing .md page return a markdown 404 that points to the index. Guessed paths such as /docs.md, /reference.md and /docs/<category>.md redirect to the matching llms.txt. Markdown requests to a project root redirect there too.
llms.txt now follows the sidebar
The llms.txt file groups pages under their sidebar categories, with child pages nested under their parents. API Reference intro pages sit under a plain API Reference heading. ReadMe replaced its llms.txt settings with a single Summarize Index option. It links the root file to per-section and per-category files instead of listing every page.
Markdown pages point agents to the index
Markdown versions of ReadMe pages now carry agentTools front matter with links to the llms.txt index and a search endpoint. The markdown versions of the API Reference Getting Started and Authentication pages now include the API’s servers and security schemes when the OpenAPI definition is visible to the reader.
Other changes in the release
ReadMe said hidden and deprecated endpoints can now be turned on one at a time for a project’s MCP server from the MCP routes settings. Each AI Agent edit suggestion, whether a page rewrite, an API Reference update or a Recipe change, has its own Apply and Revert controls. Enterprise group analytics can be exported to CSV. Translated pages can be re-translated one at a time.
Key takeaways
- Missing or guessed markdown URLs now redirect or 404 back to llms.txt.
- The llms.txt file mirrors sidebar categories, with a Summarize Index option that links to per-section files.
- Hidden and deprecated endpoints can be enabled individually for MCP.
The take
We think the redirect is the most useful change. An agent that has read one page can try /reference.md or a category path next. A dead end there leaves it with a search result or its training data. A redirect hands it the index.
The Summarize Index option has a cost. A root file that only links to section files makes an agent fetch twice before it reads a page. A large project can absorb that. A small one gains nothing from it.
We’d test a docs site the way an agent would. Request a few made-up .md paths and check where each one lands. Then see whether llms.txt still lists every page an agent should find.


