What is Applebot-Extended? AI crawler guide
Applebot-Extended is a robots.txt control token associated with Apple. See source-graded user-agent, robots.txt, verification, and observation guidance.
What is Applebot-Extended?
Applebot-Extended is a robots.txt product control token from Apple, not a fetching crawler user-agent. Robots.txt control token for whether Applebot-crawled content may be used to train Apple foundation models.
Evidence status
| Field | Value |
|---|---|
| Evidence | Officially documented |
| Lifecycle | Active |
| Purpose | training |
| robots.txt posture | honors |
| Source checked | 2026-08-11 |
Control token, not a user-agent
Applebot-Extended does not identify a separate HTTP crawler. It is used only in robots.txt policy.
Applebot-Extended collects pages for model training. Training inclusion is not the same as being cited, so measure where AI answers actually cite your site before drawing conclusions from crawl logs.
robots.txt allow example
User-agent: Applebot-Extended Allow: /
robots.txt block example
User-agent: Applebot-Extended Disallow: /
What the rule can and cannot do
Robots.txt expresses an access policy to compliant automated crawlers. It does not authenticate the sender, remove content already collected, or guarantee that a model will use or not use content.
How to verify a request
- Match the complete documented user-agent where one exists. A match is only a clue.
- Use the operator-published check: Applebot published IP ranges.
- Keep the observed request, operator documentation, and any inference as separate fields.
Observed in Trakkr connected-site data
| Measure | Value |
|---|---|
| Matching signature | Applebot-Extended |
| Classified requests | 9011 |
| Sites with matches | 20 |
| Window | 2026-07-13 through 2026-08-11 |
| Method | Raw request-signature audit across connected sites, with source-network checks. |
| Limits | Apple documents Applebot-Extended as a control token, not a fetching user-agent. None of these matching requests came from Apple network ranges, so they should not be attributed to Apple. |
JavaScript behavior
The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.
Related bots
- GPTBot: Also tracked as a training crawler.
- Google-Extended: Also tracked as a training crawler.
- KimiBot: Also tracked as a training crawler.
- Meta-ExternalAgent: Also tracked as a training crawler.
- MistralAI-Training: Also tracked as a training crawler.
- Webzio-Extended: Also tracked as a training crawler.
- anthropic-ai: Also tracked as a training crawler.
- img2dataset: Also tracked as a training crawler.
- LAIONDownloader: Also tracked as a training crawler.
- AI Training Opt-Out: Applebot-Extended is a training crawler tied to this policy decision.
- Robots.txt: Robots.txt is the control file used to allow or block Applebot-Extended.
- GrokBot: Also tracked as a training crawler.
Frequently Asked Questions
What is Applebot-Extended?
Applebot-Extended is a robots.txt control token, not a fetching crawler user-agent.
What user-agent does Applebot-Extended use?
Applebot-Extended is not an HTTP user-agent.
Can Applebot-Extended be blocked in robots.txt?
The record identifies Applebot-Extended as the token to review. Robots.txt is a request policy, not proof of model use or non-use.
How can I verify a Applebot-Extended request?
Start with the full user-agent, then use applebot published ip ranges from the operator. A user-agent match alone is not proof.
Data & Sources
- Apple documentation - Primary source for Applebot-Extended crawler details.
- Applebot-Extended verification: Applebot published IP ranges - Genuine Apple crawling comes from Apple's own network and resolves under applebot.apple.com. Requests carrying Applebot-Extended as a fetching user-agent do not.