AAI2Bot

What AI2Bot does

A training crawler collects pages to train its operator's AI models. Its visits answer no one's question in the moment: what it reads goes into the data the models learn from.

How to recognize it

AI2Bot names itself in its user agent with the token AI2Bot. That's what Ethogram matches, as a whole word in any case.

A user agent is only a claim, though: anyone can send AI2Bot. Scrapers borrow well-known names to get past rules meant for the real agent, so before you trust a visit, or block one, check where it came from.

How to tell it's really AI2Bot

Ai2 publishes no IP ranges, hostnames or signing keys for AI2Bot that Ethogram knows of, so there's nothing to check a visit against. Ethogram counts its visits as unverified: they may well be real, but the user agent is all there is.

Ethogram runs these checks on every request and gives each visit a verdict: verified, unverified or spoofed. Spoofed visits are counted apart and never added to AI2Bot's numbers. How verdicts work.

Allow or block AI2Bot in robots.txt

robots.txt groups its rules by user-agent token. To ask AI2Bot to stay off your whole site, add this group:

To let it read everything, name it with an allow rule. A group that names AI2Bot replaces the User-agent: * group for it, so it follows only these rules:

robots.txt is a request, not a lock: it's up to AI2Bot to read it and follow it. Ethogram reads your robots.txt every day and shows whether AI2Bot respects it, with the disallowed paths it asked for. To actually keep it out, use your firewall or bot protection, and check the visit's verdict first so you block impostors, not the real AI2Bot.

See what AI2Bot does on your site

Most analytics filter AI2Bot out as a bot. Ethogram records it on your server, where every request lands, with no script to load, and shows:

• Every visit from AI2Bot, with a verdict on whether it's really AI2Bot.
• Each visit step by step: the pages it read, what it searched for, and where it was blocked, looped or hit an error.
• Whether it respects your robots.txt, with the disallowed paths it asked for.
• How much of your bandwidth it takes, next to its share of requests.
• New versions of its user agent as they appear, and what changed in its behavior with each.
• A heads-up when its visits spike or stop, or when it starts getting blocked.