Skip to content
The Docs Report
Tuesday, November 17, 2026NewsletterRSS

Fern Says Agents Now Outnumber People on the Docs Sites It Hosts

Fern's Nov. 10 traffic study says agents made 51.7% of requests on 90 of its docs sites in September, up from 13.5% in January.

3 min read

FernAIllms.txt

Isometric 3D illustration of glossy hot pink boxy retro robots with square heads and antennas on a tiled floor

AI agents made 51.7% of requests to the same 90 Fern-hosted docs sites in September, up from 13.5% in January, Fern said in a Nov. 10 study of nine months of traffic. Across every site it hosts, Fern counted agent requests rising from 1.3 million a month to 7.9 million while human page loads slipped from 5.9 million to 5.1 million.

Key takeaways

  • Agents made 60.6% of all requests to Fern-hosted docs in September, Fern said.
  • Counting only requests whose user agent names an AI agent or crawler, the share was 32.8%, up from 2.3% in January.
  • 93% of agent requests went to Markdown rather than HTML.
  • Claude Code sent 1.03 million requests in September, 3.4 times the next named agent.

Agents read the API reference and the changelog

Agents accounted for 74% of page views on API reference and changelog pages, Fern said. Agents spent 31% of their reading on the API reference, against 18% for people, and read the changelog twice as much. The share varied widely by site: among 89 sites with at least 5,000 human page loads in September, agents made up less than 10% of traffic on some and more than 90% on others. Fern said agent share runs highest on docs for APIs that developers call from code.

The headline number rests on a route rule

Fern defined an agent request as a server-side request for a Markdown route or llms.txt, and a human page load as a browser page view from client-side analytics. It said 68% of September’s agent requests carried a browser or generic user agent, a group that includes scripts, curl, unidentified agents and some people who opened a Markdown page. Fern excluded 17 million internal and monitoring requests in September, more than double the agent count, and said ad blockers hide some human visits.

llms.txt grew but shrank as a share

Fern-hosted sites served llms.txt 448,667 times in September, up from about 380,000 in January, the study said. Its share of agent requests fell from 29% to 5.7% as Markdown page requests grew about eightfold. Claude Code asked for the file 32,812 times, more than GPTBot, ClaudeBot and OAI-SearchBot combined. Fern said it stopped generating llms-full.txt, which now redirects to llms.txt, because one file holding a whole site was too large for most context windows.

The take

We think the 32.8% figure is the one to quote. Fern counted any server-side request for a Markdown page as an agent, and 68% of those requests named no agent at all. Plenty of software that isn’t a coding agent fetches Markdown. The larger number measures demand for Markdown. That is useful to know, but it is a different claim from “more agents than people.”

Even the smaller number is a big change in nine months. Agents go to the reference and the changelog, and they ask for Markdown, so those pages deserve the next round of work. Check that every endpoint has a working example, every changelog entry names the endpoint and date, and every page has a .md version. Then pull a month of logs and count agent traffic two ways, by route and by named user agent, so the team knows which number it is looking at.