What is Meta-ExternalFetcher? AI crawler guide
Meta-ExternalFetcher is a crawler or fetcher associated with Meta. See source-graded user-agent, robots.txt, verification, and observation guidance.
What is Meta-ExternalFetcher?
Meta-ExternalFetcher is a documented Meta crawler or fetcher. Meta user-requested fetcher for AI and link features across Meta products.
Evidence status
| Field | Value |
|---|---|
| Evidence | Officially documented |
| Lifecycle | Active |
| Purpose | interaction |
| robots.txt posture | partial |
| Source checked | 2026-06-11 |
Documented user-agent
meta-externalfetcher/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/web-crawlers)
Meta-ExternalFetcher fires when a real user's question pulls in your page. To know how often Meta AI reaches for your brand at all, track your brand mentions in Meta AI.
robots.txt allow example
User-agent: Meta-ExternalFetcher Allow: /
robots.txt block example
User-agent: Meta-ExternalFetcher Disallow: /
What the rule can and cannot do
Robots.txt expresses an access policy to compliant automated crawlers. It does not authenticate the sender, remove content already collected, or guarantee that a model will use or not use content.
How to verify a request
- Match the complete documented user-agent where one exists. A match is only a clue.
- Look for operator-published IP ranges, reverse DNS, or signed-request guidance. None is attached to this record.
- Keep the observed request, operator documentation, and any inference as separate fields.
Observed in Trakkr connected-site data
| Measure | Value |
|---|---|
| Matching signature | Meta-ExternalFetcher |
| Classified requests | 2781 |
| Sites in sample | 85 |
| Window | 2026-07-18 through 2026-08-17 |
| Method | Finalized daily crawler summaries from connected Trakkr sites, grouped by the crawler signature detected in each request. |
| Limits | This is a connected-site sample, not a representative sample of the web. A matching user-agent or signature does not prove that the named operator sent the request. Counts describe classified request signatures, not market share or unique pages crawled. |
JavaScript behavior
The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.
Related bots
- Google-GeminiNotebook: Also tracked as a live fetcher.
- MistralAI-User: Also tracked as a live fetcher.
- ChatGPT-User: Also tracked as a live fetcher.
- Claude-Code: Also tracked as a live fetcher.
- cohere-ai: Also tracked as a live fetcher.
- Google-Gemini-CLI: Also tracked as a live fetcher.
- Kimi-User: Also tracked as a live fetcher.
- ChatGPT Agent: Also tracked as a live fetcher.
- Perplexity-User: Also tracked as a live fetcher.
- Meta AI: Meta-ExternalFetcher connects this operator term to its crawler behavior.
- Noindex: Meta-ExternalFetcher gives crawler context for Noindex.
Frequently Asked Questions
What is Meta-ExternalFetcher?
Meta-ExternalFetcher is a officially documented crawler or fetcher record associated with Meta.
What user-agent does Meta-ExternalFetcher use?
meta-externalfetcher/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/web-crawlers). Check the evidence status before attributing a matching request.
Can Meta-ExternalFetcher be blocked in robots.txt?
The record identifies Meta-ExternalFetcher as the token to review. Robots.txt is a request policy, not proof of model use or non-use.
How can I verify a Meta-ExternalFetcher request?
Treat the user-agent as a clue and look for operator-published IP, reverse-DNS, or signature guidance.
Data & Sources
- Meta documentation - Primary source for Meta-ExternalFetcher crawler details.