What is Google-Agent? AI crawler guide
Google-Agent is a crawler or fetcher associated with Google. See source-graded user-agent, robots.txt, verification, and observation guidance.
What is Google-Agent?
Google-Agent is a documented Google crawler or fetcher. Google user-triggered fetcher used by agents that act on a person's request, including Project Mariner.
Evidence status
| Field | Value |
|---|---|
| Evidence | Officially documented |
| Lifecycle | Active |
| Purpose | interaction |
| robots.txt posture | partial |
| Source checked | 2026-08-18 |
Documented user-agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36; compatible; Google-Agent
Google-Agent access decides whether Google can fetch your pages. Whether those pages get used in generated answers is a separate question, so track your presence in Google's AI results rather than treating crawl volume as visibility.
robots.txt allow example
User-agent: Google-Agent Allow: /
robots.txt block example
User-agent: Google-Agent Disallow: /
What the rule can and cannot do
Robots.txt expresses an access policy to compliant automated crawlers. It does not authenticate the sender, remove content already collected, or guarantee that a model will use or not use content.
How to verify a request
- Match the complete documented user-agent where one exists. A match is only a clue.
- Use the operator-published check: Published user-triggered fetcher IP ranges.
- Keep the observed request, operator documentation, and any inference as separate fields.
JavaScript behavior
The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.
Related bots
- Google-Gemini-CLI: Another Google live fetcher to compare.
- cohere-ai: Also tracked as a live fetcher.
- Claude-Code: Also tracked as a live fetcher.
- Google-GeminiNotebook: Another Google live fetcher to compare.
- Gemini-Deep-Research: Another Google live fetcher to compare.
- Google-NotebookLM: Another Google live fetcher to compare.
- AmazonBuyForMe: Also tracked as a live fetcher.
- ChatGPT Agent: Also tracked as a live fetcher.
- Claude-Web: Also tracked as a live fetcher.
- Gemini: Google-Agent gives crawler context for Gemini.
- Gemini 2.0: Google-Agent gives crawler context for Gemini 2.0.
Frequently Asked Questions
What is Google-Agent?
Google-Agent is a officially documented crawler or fetcher record associated with Google.
What user-agent does Google-Agent use?
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36; compatible; Google-Agent. Check the evidence status before attributing a matching request.
Can Google-Agent be blocked in robots.txt?
The record identifies Google-Agent as the token to review. Robots.txt is a request policy, not proof of model use or non-use.
How can I verify a Google-Agent request?
Start with the full user-agent, then use published user-triggered fetcher ip ranges from the operator. A user-agent match alone is not proof.
Data & Sources
- Google documentation - Primary source for Google-Agent crawler details.
- Google-Agent verification: Published user-triggered fetcher IP ranges - Google also documents experimental Web Bot Auth signatures for a subset of requests. Not every request is signed.