What is Claude-User? AI crawler guide
Claude-User is a crawler or fetcher associated with Anthropic. See source-graded user-agent, robots.txt, verification, and observation guidance.
What is Claude-User?
Claude-User is a documented Anthropic crawler or fetcher. Anthropic user-triggered fetcher for Claude answers that need a specific web page.
Evidence status
| Field | Value |
|---|---|
| Evidence | Officially documented |
| Lifecycle | Active |
| Purpose | interaction |
| robots.txt posture | honors |
| Source checked | 2026-08-18 |
Documented user-agent
Claude-User
Claude-User fires when a real user's question pulls in your page. To know how often Claude reaches for your brand at all, track your brand mentions in Claude.
robots.txt allow example
User-agent: Claude-User Allow: /
robots.txt block example
User-agent: Claude-User Disallow: /
What the rule can and cannot do
Robots.txt expresses an access policy to compliant automated crawlers. It does not authenticate the sender, remove content already collected, or guarantee that a model will use or not use content.
How to verify a request
- Match the complete documented user-agent where one exists. A match is only a clue.
- Use the operator-published check: Published IP ranges.
- Keep the observed request, operator documentation, and any inference as separate fields.
JavaScript behavior
The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.
Related bots
- DuckAssistBot: Also tracked as a live fetcher.
- Gemini-Deep-Research: Also tracked as a live fetcher.
- MistralAI-User: Also tracked as a live fetcher.
- cohere-ai: Also tracked as a live fetcher.
- Amzn-User: Also tracked as a live fetcher.
- ChatGPT-User: Also tracked as a live fetcher.
- Kimi-User: Also tracked as a live fetcher.
- Perplexity-User: Also tracked as a live fetcher.
- Claude-Code: Another Anthropic live fetcher to compare.
- Anthropic: Claude-User connects this operator term to its crawler behavior.
- Claude 3.5 Sonnet: Claude-User gives crawler context for Claude 3.5 Sonnet.
- SemrushBot-OCOB: Shares crawler-policy context with Claude-User.
Frequently Asked Questions
What is Claude-User?
Claude-User is a officially documented crawler or fetcher record associated with Anthropic.
What user-agent does Claude-User use?
Claude-User. Check the evidence status before attributing a matching request.
Can Claude-User be blocked in robots.txt?
The record identifies Claude-User as the token to review. Robots.txt is a request policy, not proof of model use or non-use.
How can I verify a Claude-User request?
Start with the full user-agent, then use published ip ranges from the operator. A user-agent match alone is not proof.
Data & Sources
- Anthropic documentation - Primary source for Claude-User crawler details.
- Claude-User verification: Published IP ranges - Anthropic says IP blocking is not a durable opt-out because it can prevent the crawler from reading robots.txt.