Field notes
AI agents are already in your traffic. Your analytics can't see them.
Answer engines, crawlers and browser agents visit your site every day. Most never run your analytics tag, and the few that do get filtered out or counted as people. Here's why, and what to measure instead.
Someone asks ChatGPT which trail jacket to buy for a wet autumn. To answer, it fetches your product page, reads the specs and the price, and quotes them back. An hour later, a personal agent working for someone else compares the same jacket across three stores, adds one to a cart and tries to check out.
Both visits happened on your website. Neither shows up in your analytics as a visitor.
This isn't a niche problem anymore. People now hand a growing share of their browsing to AI: asking an assistant instead of searching, letting an agent fill in the form, sending a personal agent to do the comparison shopping. Every one of those tasks turns into requests to real websites. The visitor is new. The tools we use to understand visitors are not.
Why classic analytics misses agents
Web analytics was built for one kind of visitor: a person, in a browser, on a page. Agents break that assumption in three different ways.
Most agents never run your tag
Tag-based analytics works by running JavaScript in the visitor's browser. The script loads, records a pageview and sends it home. No script, no pageview.
Answer engines and crawlers usually don't run your JavaScript at all. They fetch the HTML, extract what they need and leave. From your analytics' point of view, that visit never happened. Your server answered the request and your CDN billed you for it, but your dashboard stayed flat.
The ones that do run it get filtered, or counted as people
Browser agents are different. They drive a real browser, so your tag does fire. What happens next depends on how they identify themselves.
If the agent announces itself as a known bot, most analytics tools drop it on purpose. Bot filtering exists to keep human metrics clean, and it does that job well. It just throws the agent visit away instead of telling you about it.
If the agent looks like an ordinary browser, it gets counted as a person. Now it quietly distorts your human metrics: sessions with no scrolling, unusually fast form fills, sudden bounces at a checkout step, conversion rates that move for no reason anyone on the team can find.
Either way, you lose the information you actually wanted: that an agent came, and what happened to it.
The questions are different
Even with perfect detection, classic reports would struggle. They answer human questions: where did people come from, how long did they stay, where in the funnel did they drop off.
Agent visits raise different questions:
- Who sent it? An answer engine answering one question, a crawler building an index, or a browser agent acting for a specific person.
- What was it trying to do? Read a price, compare products, book a table, sign up.
- Did it get there? And if not, what stopped it: a CAPTCHA, a login wall, a control it couldn't use.
A pageview count can't answer any of these. Bounce rate means little for a visitor that was only ever going to read one page.
What you're missing when you can't see them
The cost of invisible agent traffic isn't an abstract gap in a chart. It shows up as decisions made without the facts.
Your content is being read and quoted without you knowing which parts. When an answer engine summarizes your pricing or your returns policy, it uses whatever it could read from the HTML. If the price is rendered by JavaScript after load, the agent may never see it, and the answer it gives may be incomplete or out of date.
Tasks are failing silently. When a person gets stuck at checkout, some of them email support or try again later. When an agent gets stuck, it usually retries a few times, gives up and tells its person it couldn't finish. Your server logged a series of successful requests. Nobody on your side ever hears about the failure.
Releases break things for agents that work fine for people. A redesigned form, a new cookie banner or a custom dropdown can be perfectly usable with a mouse and impossible for an agent. Your human conversion rate stays steady, so nothing looks wrong.
You can't make policy decisions with confidence. Should you allow training crawlers? Welcome AI search crawlers? Rate-limit anything that isn't a browser? Each choice has real trade-offs, and making them blind means you either block the visitors that send you customers or pay for traffic that gives you nothing back.
How big is it on your site?
The honest answer: it depends, and you should measure it rather than trust anyone's average.
The share of agent traffic varies a lot with what you publish. Documentation, product catalogs and pricing pages get read by answer engines constantly. Stores and booking flows see more browser agents. Content-heavy sites see more crawlers. The mix also changes quickly as new agents launch and old ones change how they identify themselves.
On our homepage we follow a fictional store, shop.example.com, where agents make up about one visit in eight. That number is illustrative. Yours could be lower or much higher, and the interesting part is rarely the total. It's the breakdown: which agents, doing what, and how often they succeed.
What you can do today
You don't need a new tool to start looking. A few steps will tell you a lot.
- Read your server or CDN logs. Search the user agent strings for names like GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot and Bytespider. You'll see the crawlers and answer engines your tag never recorded.
- Don't trust user agents alone. Anyone can claim to be GPTBot. Most major AI companies publish the IP ranges their crawlers use, so check claimed bots against them. Some agents now also sign their requests, which lets you verify them cryptographically.
- Load your key pages without JavaScript. Product pages, pricing, docs, policies. Whatever is missing there is missing for most answer engines too.
- Try a task with an agent yourself. Ask a browser agent to buy something, book something or sign up on your site, and watch where it hesitates.
This gets you a snapshot. What it won't give you is the ongoing picture: every agent visit, named, with what it did step by step and where it got stuck.
Measuring agents on their own terms
That's the gap Ethogram fills. It runs on your server, from a few lines of Next.js, Express or Cloudflare Worker code, and combines the signals above (user agent, verified IP ranges, signed requests and behavior) into one verdict per visit: verified, unverified or spoofed. Every agent gets a name and a type: answer engine, search crawler, training crawler or browser agent.
Then it records what the agent did as a sequence of behaviors: navigate, read, click, fill a field, retry, complete the task, hand it back, get blocked. That record is an ethogram, the catalog ethologists use to describe what a species does. It lets you count the behaviors that matter, compare them before and after a release, and get alerted when something changes.
Agents are a new kind of visitor. They deserve to be seen on their own terms, not filtered away or mistaken for people.
- agent traffic
- analytics