AI Crawler Directory · Amazon
Amazonbot:
what it is and how to verify it.
OperatorAmazon
PurposeAlexa answers + AI improvement
Respects robots.txtYes
Executes JavaScriptNo
Hits verifiableYes — published IPs
TypeHTTP crawler / fetcher
Amazonbot is Amazon’s web crawler. It gathers content to improve Amazon’s products and services — most visibly Alexa’s question answering — and Amazon states the data may be used to train its AI models. It sits between a classic search indexer and a training crawler.
It is a well-behaved bot by documentation: Amazon commits to the Robots Exclusion Protocol, honors allow/disallow directives (caching robots.txt for up to 30 days), respects rel=nofollow, and supports the noarchive meta tag as an opt-out from model-training use — a page-level control most AI crawlers lack.
Amazon publishes Amazonbot’s IP addresses for verification, so hits can be authenticated rather than taken on faith. With Alexa+ features built on generative AI, Amazonbot’s crawl is Amazon’s pipeline from the open web into those answers.
Amazonbot User-Agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)How to verify Amazonbot hits are real
The User-Agent header is plain text — any client can claim to be Amazonbot, and scrapers routinely do. Amazon publishes Amazonbot’s IP addresses on its developer site; authentic hits fall inside them.
https://developer.amazon.com/amazonbot/ip-addresses/Fetch the list and check the hit’s source IP against it — for a quick manual look:
curl -s https://developer.amazon.com/amazonbot/ip-addresses/Allow or block Amazonbot in robots.txt
This is a search-index bot: it feeds the citations AI answers link to. If you want AI visibility — and the referral traffic behind it — the standard posture is to allow it.
Allow
User-agent: Amazonbot
Allow: /Block
User-agent: Amazonbot
Disallow: /Amazonbot — frequently asked questions
What is Amazonbot?
Amazonbot is Amazon’s web crawler. It collects content to improve Amazon products and services — including Alexa answers — and Amazon states crawled data may help train its AI models.
Does Amazonbot respect robots.txt?
Yes. Amazon documents compliance with the Robots Exclusion Protocol, including user-agent-specific allow/disallow rules. Note it may use a cached robots.txt for up to 30 days, so rule changes are not instant.
Can I keep Amazonbot from training on my content without blocking it?
Yes — Amazon honors the noarchive robots meta tag as a signal to exclude a page’s content from model-training use while still allowing the crawl. Few AI crawlers offer this page-level split.
How do I verify Amazonbot hits?
Amazon publishes Amazonbot’s IP addresses at developer.amazon.com/amazonbot (IP addresses page). Check the source IP against that list; the UA string alone can be forged.
Related crawlers
Related reading
See which AI crawlers hit your site — and what the traffic earns
Attrifast tracks AI referrals as revenue lines: ChatGPT, Perplexity, Claude, and Gemini visits tied to real Stripe money. $9.99/mo flat.
- One script tag and a Stripe key — live in minutes
- Cookieless, so no consent banner for analytics
- Every AI referral matched to the payment it produced
7-day free trial · $0 due today · then $9.99/mo · cancel anytime
